diff --git a/CLAUDE_TODO_2026-08-29.md b/CLAUDE_TODO_2026-08-29.md new file mode 100644 index 0000000000000..8b80860d6326a --- /dev/null +++ b/CLAUDE_TODO_2026-08-29.md @@ -0,0 +1,96 @@ + +## P1 provisional/committed 持久化（追加） + +- 在发布 Fenced 前持久化 provisional v38；失败则绝不发布 Fenced。 +- 全体 provisional Fence 收敛后持久化 committed v38；失败保持 ingress/DDL fail-closed。 +- restart 对 committed 执行 startup frontier fence 后开放；对 provisional 恢复 v38 capability 但保持 ingress/public DDL 关闭，等待完整 activation retry。 +- restart regression 必须创建新 service 并调用 initMetadata()，另覆盖 provisional/committed persist failure。 + +## 2026-08-30：最新主干冲突与 review comments + +1. Merge 最新 `mo/main`，保留主干 v38/v39 语义，将 DDL visibility protocol 完整迁移到下一个版本 v40。 +2. 在 ctl dispatch 前读取 raw CN inventory，要求 activation target 精确覆盖全部仍可能公开产出 DDL 的 CN，并校验 generation/address 与 receiver capability。 +3. 在 startup ingress publication 前重新读取权威 cluster epoch，关闭 markerless join 与最终 epoch commit 之间的 TOCTOU；若 cut 已提交则转为 v40 fail-closed，不能开放 ingress。 +4. 增加版本兼容、遗漏 legacy producer、markerless join/commit 交错回归；执行相关 UT、race、vet、自审后 push，并 resolve threads/request re-review。 + +## 2026-08-30：再次解决主干冲突 + +1. Merge 最新 `mo/main`，按主干协议版本与生成文件保留双方语义。 +2. 重新生成受影响生成文件，运行冲突闭包测试与 `git diff --check`。 +3. Push PR 分支并确认恢复 mergeable。 + +## 2026-08-30：原子 membership/epoch commit + +1. 将 activation target 的 service ID、generation、query address 作为 epoch commit proof 传入 HAKeeper heartbeat transition。 +2. HAKeeper RSM 在同一 transition 内按 raw CNState 精确校验 membership/capability tuple；仅校验成功时推进 cluster epoch。 +3. CN 必须先获得 RSM 返回的 committed epoch，再持久化本地 committed marker 和开放 ingress；拒绝时保持 provisional/fail-closed。 +4. 增加 final scan 后、epoch commit 前 markerless join 的确定性回归，并运行相关 UT/race/vet。 + +## 2026-08-30：frontend txn 测试冲突 + +1. 合并最新 `mo/main`，在 `pkg/frontend/txn_test.go` 同时保留 DDL sync 回归与主干 panic rollback 回归。 +2. 运行 frontend 冲突闭包测试、vet 和 diff check，随后 push。 + +## 2026-08-30：最新 deep-review P1 与设计 gate + +1. markerless 首次升级 CN 保留当前主干已部署协议 baseline（至少 v40），仅将 v41 DDL activation 状态保持未完成/fail-closed，不回退共享 scalar 到 v37。 +2. HAKeeper monotonic epoch 已达 v41 后拒绝任何局部低版本 downgrade，避免现有会话继续作为无 fan-out DDL producer。 +3. 增加 baseline preservation、post-cut downgrade rejection 回归。 +4. 新增稳定版本化设计文档，说明状态机、rollout/rollback、direct/proxy ingress、复杂度/延迟、观测与恢复；补充确定性 two-CN 自动 DDL create/read 公共路径测试。 +5. 合并最新主干，执行相关 UT/race/vet/self-review，push 并请求 re-review。 + +## 2026-08-30：v41 公共路径回归与最新 review + +1. 查询并分析 PR 最新 inline/general review comment，更新变更闭包与验收条件。 +2. 修复 two-CN 公共路径测试发现的生产激活协调阻塞：保持 activation/startup/shutdown 串行所有权，同时允许心跳在长阶段等待期间发布 Prepared/Fenced 状态，避免用重试或弱化断言掩盖问题。 +3. 通过生产 `mo_ctl SetProtocolVersion` 激活精确 CN 集合，并用故障注入证明 CREATE DDL 实际经过 v41 `SyncCommitV2`，随后验证另一 CN 首次读取可见。 +4. 补齐心跳/RSM/超时/并发 UT，运行生成、focused/owning package、race、vet、embedded topology 与 self-review；通过后 commit/push、处理 review thread 并请求 re-review。 + +## 2026-08-30：处理新一轮 review comments + +1. 获取 exact-head 最新 review 与 unresolved threads，逐条复现并确认根因。 +2. 对确认问题做最小完整修复，补充对应确定性 UT/拓扑回归。 +3. 运行受影响 owning package、race、vet、生成文件与 diff 检查。 +4. commit/push，逐条英文回复并 resolve，重新请求 review。 + +进度（2026-08-31）： +- 已将 DDL frontier 提升为 HAKeeper RSM 中独立于 CN store 生命周期的单调集群最大值；activation/restart 先从 txn client 重建 durable high-water mark，再发布并同步该全局值。 +- 已增加 producer replacement/store removal 不得清空 frontier 的 RSM 回归，以及 replacement 后 activation 仍等待旧 frontier 的 CN 回归。 +- 按用户决定不再等待其他并发 PR：本 PR 固定使用当前无人占用的 MORPC v43，并为两个已在 review 的并发 owner 预留 v41/v42；同步更新实现、测试、设计与生产路径回归。 + +## 2026-08-31：解决最新主干冲突 + +1. fetch 并 merge 最新 `mo/main`，逐文件确认双方语义，保留主干新增能力与本 PR v43 DDL fence。 +2. 重新生成必要生成文件，运行冲突相关 owning package、embedded 双 CN、race/vet 与 diff 检查。 +3. commit/push merge 结果，确认 PR mergeability 与 unresolved thread 状态。 + +## 2026-08-31：修复最新 CI failure + +1. 获取 PR exact head 的失败 check/job 原始日志，区分代码回归与基础设施问题并定位最小根因。 +2. 对代码问题补充确定性回归并修复；不使用重试或弱化断言掩盖失败。 +3. 运行 owning package、相关 race/vet/embedded 验证，commit/push 后确认新 CI 状态。 + +## 2026-09-01：frontier publication generation fence + +1. 同步 frontier heartbeat 必须验证 `CommandBatch.ViewMetadataAdmission.Generation` 与本地 incarnation generation 完全一致。 +2. HAKeeper 返回更高 authoritative generation 时立即 revoke 旧 incarnation 并向 frontend 返回错误，禁止继续 fan-out/成功确认。 +3. 增加 generation N commit → N+1 takeover → N publication rejected → N crash → restart 仅同步旧 durable frontier 的确定性回归，并运行 owning package/race/vet。 + +## 2026-09-01：generation revocation TaskRunner self-wait + +1. 将 revocation 拆为同步不可逆 admission seal 与异步 frontend/TaskRunner/full-service drain。 +2. 同步 frontier rejection 只等待 seal，不得等待可能包含当前 SQL DDL 调用栈的 TaskRunner.Stop。 +3. 增加 TaskRunner SQL DDL 自身发现 stale generation、rejection 先返回、task 退出后异步 Stop 才完成的确定性 Q2 回归，并运行 race/vet。 + +## 2026-09-01：revoked TaskRunner 与正常 Close 协调 + +1. 在 `s.task` 下建立 detached revoked-runner 的共享 ownership/completion；异步 revocation drain 是唯一 Stop owner。 +2. 正常 `stopTask` 在锁内转移普通 runner ownership，锁外执行 holder.Close/runner.Stop；若 runner 已由 revocation 摘除，则等待共享 completion 后才允许关闭下游依赖。 +3. 分别覆盖 stopTask 先取得 runner 与 revocation 先摘除 runner 两种顺序，证明无持锁等待、无 double Stop、无依赖提前关闭。 + +## 2026-09-01：新增 review blockers（frontier ack / heartbeat response / v44） + +1. 将 DDL capability 从与 #27553 冲突的 v43 移至下一未占用 v44，并迁移全部 gate、method map、测试、`mo_ctl` 路径及设计文档。 +2. 在同一 HAKeeper RSM heartbeat 响应中返回应用后的 authoritative frontier；CN 仅在 generation 精确匹配且 frontier >= T 时承认同步发布，旧 RSM 静默丢字段时 fail closed。 +3. 所有 direct CN heartbeat 将 CommandBatch 交给现有 command mutex/dedupe owner，避免 command-delivery-disabled 阶段的 destructive schedule command 丢失。 +4. 增加旧 HAKeeper 丢 frontier 与 legacy command delivery 的确定性回归，运行 affected package、race、vet 和 protobuf 测试。 diff --git a/docs/design/CLAUDE_cross_cn_ddl_visibility_fence.md b/docs/design/CLAUDE_cross_cn_ddl_visibility_fence.md new file mode 100644 index 0000000000000..c73715c3f7cc3 --- /dev/null +++ b/docs/design/CLAUDE_cross_cn_ddl_visibility_fence.md @@ -0,0 +1,189 @@ +# Cross-CN DDL Visibility Fence + +- Status: **Approved** +- Revision: 4 +- Approval: user approval recorded in the PR implementation session on 2026-08-30 +- Owning issue: #27743 +- Implementation PR: #27756 +- Protocol version: MORPC v44 + +## 1. Classification and motivation + +This change is a distributed-protocol feature rather than a narrow local fix. It crosses frontend transaction commit, CN lifecycle and query RPC, cluster membership, HAKeeper replicated state, protobuf compatibility, and proxy admission. It changes persistent state, wire contracts, mixed-version rollout, restart, and rollback behavior; therefore it requires design approval. + +Issue #27743 demonstrates a missing catalog-visibility invariant: a DDL can commit on CN A while CN B admits a fresh snapshot before applying the corresponding catalog logtail. The client then observes `no such table` or equivalent stale-catalog behavior. + +## 2. Invariant and success criteria + +### Safety invariant + +After the v44 deployment epoch is committed, every CN capable of public DDL production must satisfy both conditions: + +1. its runtime protocol is at least v44, so DDL commit performs `SyncCommitV2` fan-out; and +2. it belongs to the exact generation/address membership set atomically committed by HAKeeper, or it remains fail-closed until it joins a later exact cut. + +Before a CN opens public ingress, its local catalog frontier must be at least the maximum frontier reported by all participants in the applicable cut. + +### Negation + +It is unsafe for any public or already-connected CN to commit DDL below v44 while another public CN can admit a snapshot that has not applied that commit. + +### Measurable criteria + +- A create on CN0 followed immediately by a first read/load on CN1 succeeds without explicit `SYNCCOMMIT`. +- Markerless startup preserves the current-main v42 baseline before the v44 cut. +- No local downgrade below v44 is accepted after the monotonic v44 epoch is committed. +- Membership change between final scan and epoch commit rejects the old target set atomically. +- Restart, timeout, persistence failure, response loss, and leader failover remain fail-closed. + +## 3. State ownership + +| State | Owner | Persistence | Meaning | +|---|---|---|---| +| Compiled latest protocol | binary/runtime | binary | Receiver/sender code capability only | +| Local deployed protocol | CN metadata file | durable local FS | This CN completed or provisionally entered a cut | +| Cluster deployed epoch | HAKeeper CNState | replicated snapshot/log | Monotonic cluster-wide committed cut | +| Admission generation/address | HAKeeper CNState | replicated snapshot/log | Identity of one CN incarnation | +| Prepared/Fenced/Complete | CN service and HAKeeper CNState | heartbeat-replicated phase proof plus local marker | Progress bound to one authoritative CN incarnation | +| Last committed DDL frontier | DDLCommitGate and HAKeeper CNState | monotonic cluster-lifetime maximum replicated in HAKeeper state | Durable activation high-water mark that survives CN restart, replacement, and store removal | +| Public DDL gate | frontend `DDLCommitGate` | process-local | Whether new public/background DDL may enter | +| Proxy ingress readiness | CN heartbeat/HAKeeper | replicated latest state | Whether new routed sessions may enter | + +HAKeeper is the first owner of cluster epoch and membership linearization. CN local metadata is not authoritative for cluster membership. + +## 4. Protocol states + +A CN is in one of these logical states: + +1. **Baseline v42**: v44 not deployed; existing v38-v42 contracts remain active. Public ingress may be open. +2. **Withdrawing**: activation blocks new DDL, withdraws ingress, and drains active DDL. +3. **Prepared v44**: local old-protocol producers are drained; runtime can receive v44 RPCs. +4. **Provisionally fenced**: local frontier synchronization completed and `-44` is durable; ingress remains closed. +5. **Cluster committed**: HAKeeper atomically validated exact `(serviceID, generation, queryAddress)` membership and advanced epoch to 43. +6. **Locally committed**: CN persisted `43`; only then may it republish ingress and unblock DDL. +7. **Markerless post-cut**: no local marker but HAKeeper epoch is 43; runtime remains v44 and ingress/DDL remain closed until a complete retry. + +The cluster epoch commit is the linearization point. Prepared, Fenced, and the last committed DDL frontier are published through each incarnation's heartbeat. The commit heartbeat contains the exact target tuples. In one replicated transition HAKeeper updates the sender heartbeat, compares all eligible raw CNState members, exact generation/address, receiver capability, and that each current incarnation itself published Prepared and Fenced, then advances the epoch only on exact equality. A replacement cannot reuse an older incarnation's Fenced proof. A join before this transition invalidates the target set; a join after it observes epoch 44 and is rejected as ingress-ready. + +## 5. End-to-end flow + +### First rollout + +1. All LogStore/HAKeeper replicas advertise support for the epoch schema. +2. Every CN runs v44-capable code but markerless CNs keep protocol baseline v42. +3. `mo_ctl SetProtocolVersion` refreshes raw authoritative CN membership. +4. The requested set must exactly match all eligible CN tuples and each target must advertise the v44 receiver/barrier capability. +5. Targets concurrently withdraw ingress, block and drain DDL, capture their reconstructed latest committed timestamp, then heartbeat Prepared together with that frontier. HAKeeper advances a cluster-lifetime maximum and never lowers it when an incarnation restarts, is replaced, or is removed. +6. Each target reads and applies the replicated cluster-lifetime frontier, durably enters provisional Fenced, and heartbeats Fenced. This avoids cyclic QueryService control RPCs while every target is already serving a long-running activation RPC. +7. Each target confirms all exact current incarnations are Fenced in HAKeeper; HAKeeper atomically revalidates those tuple-bound proofs and commits epoch 44. +8. Each CN persists local committed 43, republishes ingress if listeners are live, and unblocks public DDL. + +### Steady-state DDL + +A public real-user DDL, and background DDL after public listeners are enabled, enters `DDLCommitGate`. After commit, the producer synchronously advances the monotonic HAKeeper cluster frontier before acknowledging success; publication failure fails the statement closed. The returned `CommandBatch.ViewMetadataAdmission.Generation` must exactly equal the sender generation, and the returned authoritative frontier must be at least the exact T submitted in that transition. A stale sender or a legacy HAKeeper RSM that silently ignores the frontier therefore cannot mistake RPC delivery for durable frontier acceptance. Every direct CN heartbeat transfers its returned `CommandBatch` to the periodic heartbeat's command mutex/deduplication owner before returning, so startup, ingress, activation, withdrawal, and frontier publication cannot destructively consume and discard a legacy schedule command. TaskRunner, frontend, and full-service drains run asynchronously after the seal: a TaskRunner SQL DDL may itself discover the rejection, so synchronously waiting for that runner would self-deadlock. Revocation transfers a detached runner into shared drain ownership with a completion channel; normal `service.Close` waits for that completion before closing task/txn/RPC dependencies. Conversely, if normal Close takes runner ownership first, it releases the task mutex before `runner.Stop`, allowing the SQL task to seal revocation and exit. This removes both the commit-success-to-periodic-heartbeat crash window and the old-incarnation-after-takeover window without adding a revocation wait cycle. Protocol v44 then triggers `SyncCommitV2` to all barrier-ready CNs. The operation succeeds only after generation-bound durable frontier publication and required receivers have applied/synchronized the commit frontier. Bootstrap background work before ingress remains exempt to avoid depending on an unavailable HAKeeper/QueryService. + +### Scale-out and replacement + +A markerless CN performs an atomic ingress heartbeat handshake. If it linearizes before the cluster commit, it becomes authoritative membership and invalidates any old target proof. If it linearizes after commit, HAKeeper forces ingress false and returns epoch 44. It never becomes a public v42 producer after the cut. + +## 6. Failure, retry, and lifecycle behavior + +- **Withdrawal failure or drain timeout**: ingress and DDL gate remain closed; retry uses raw QueryService identity. +- **Frontier RPC/application failure**: CN remains Prepared or provisional and closed. +- **Provisional persistence failure**: Fenced is never published. +- **Atomic epoch membership rejection**: epoch does not advance; all targets remain closed for retry with a refreshed set. +- **Epoch response loss**: epoch may be committed, but local marker remains provisional and ingress closed; retry learns the committed epoch. +- **Committed local persistence failure**: cluster epoch remains committed; this CN remains closed and restarts from provisional state. +- **Ingress publication uncertainty**: perform a bounded cleanup withdrawal; never assume publication failed. +- **Restart**: committed marker runs startup frontier synchronization before opening; provisional or markerless post-cut starts v44 fail-closed. +- **Shutdown**: stop periodic heartbeat publication, then withdraw ingress/barrier state before stopping QueryService. +- **Leader failover**: activation is gated until every voting and non-voting LogStore advertises epoch-schema capability, so any eligible HAKeeper leader preserves the field. + +Retries are idempotent for the same generation and target set. Replacement generations or addresses are rejected before local state mutation. + +## 7. Compatibility, rollout, downgrade, and rollback + +### Mixed-version baseline + +Current main already deploys v42 semantics. A fresh v44-capable process without a DDL marker therefore remains at v42, not v37. The v44 DDL state is separate from unrelated v38-v42 capabilities. + +### Rollout order + +1. Upgrade all voting and non-voting LogStores/HAKeeper replicas. +2. Upgrade every CN; verify barrier receiver capability in raw inventory. +3. Invoke one exact complete-target v44 activation. +4. Verify epoch 44 and all eligible CN ingress/committed markers. + +### Downgrade policy + +Before epoch 44, ordinary protocol changes at or below v42 remain possible. After epoch 44, local downgrade below v44 is rejected. A safe rollback would require a separately designed atomic cluster rollback that withdraws and drains every DDL producer before lowering the epoch; this revision deliberately does not implement epoch rollback. + +Binary rollback after epoch 44 is unsupported until such a rollback protocol exists. Operators must restore forward to a v44-capable binary. + +## 8. Proxy and direct ingress + +Proxy routing consumes HAKeeper admission/readiness and therefore excludes fail-closed CNs. Direct SQL listeners are additionally protected by the local `DDLCommitGate`; suppressing proxy routing alone is insufficient because existing and direct sessions survive routing changes. Listener-ready and ingress-ready are separate lifecycle facts. + +## 9. Performance and capacity + +Let `N` be eligible CN count. + +- Activation performs O(N) membership validation and scans heartbeat-replicated phase/frontier state. It is an operator-triggered bounded transition, not a per-statement hot path. Target count is capped at 1024. +- DDL commit fan-out is O(N) RPCs and O(N) response ownership. DDL is low frequency relative to DML; no unbounded queue or background worker is introduced. +- Requests are bounded by existing discovery/RPC contexts. Target maps and response slices are released after each operation. +- Expected added DDL latency is the slowest required CN frontier application plus network fan-out. Rollout acceptance should record N-CN p50/p95 for representative 3-CN and larger staging clusters; no latency claim is made without that evidence. + +## 10. Observability and operations + +Required diagnostic state is available through protocol-version RPCs, CN heartbeat inventory, local deployed marker, HAKeeper cluster epoch, admission generation/address, barrier readiness, and ingress readiness. Activation errors identify missing targets, stale generation/address, unsupported HAKeeper replicas, membership drift, persistence failure, or frontier timeout. + +Follow-up metrics should count activation attempts/failures by phase and DDL fan-out latency/failures. Until those metrics land, logs and `mo_ctl` inventory are the operational source. + +## 11. Security and abuse bounds + +The protocol adds no tenant data exposure and uses internal QueryService/HAKeeper channels. Exact generation/address checks prevent stale incarnation control. The 1024-target cap and bounded contexts limit operator-triggered amplification. Existing authorization for `mo_ctl` remains the trust boundary. + +## 12. Alternatives + +### A. Periodic or statement-local sleeps + +Rejected: timing does not establish catalog visibility, is flaky, and adds unconditional latency. + +### B. Read HAKeeper epoch once at startup + +Rejected: a join can race the final commit after the read. Re-reading at ingress remains TOCTOU unless join and commit share one replicated ordering. + +### C. Always use latest compiled scalar protocol + +Rejected: compiled capability is not proof that every producer completed the distributed cut. + +### D. Separate DDL feature epoch from the shared MORPC scalar + +Architecturally clean and avoids scalar coupling, but still requires the same membership/ingress linearization and broader API migration. This revision retains MORPC v44 as the sender/receiver capability gate while storing deployment completion separately. + +### E. TN-only global catalog barrier on every new transaction + +Could provide a stronger generic read contract, but materially changes every transaction admission path and latency. The selected approach scopes cost to DDL and activation while preserving the required immediate cross-CN visibility. + +## 13. Validation matrix + +| Contract | Deterministic evidence | +|---|---| +| v42 baseline preserved | markerless default-v44 startup UT | +| post-cut downgrade rejected | completed-v44 downgrade UT | +| exact authoritative targets | ctl raw-membership omission/capability UT | +| join/commit atomicity | CNState RSM ordering UT: final scan, join, stale commit | +| HAKeeper failover compatibility | old capability view rejection UT | +| provisional/committed persistence | injected Replace failures plus distinct-service restart UT | +| startup and live frontier fencing | mock frontier/application ordering UT and race runs | +| automatic public behavior | two-CN embedded create on CN0 followed immediately by first CN1 read without `SYNCCOMMIT` | +| cancellation and publication uncertainty | bounded timeout, stale heartbeat, cleanup withdrawal UTs | + +## 14. Decision log and open items + +- Chosen linearization point: HAKeeper replicated heartbeat transition with exact tuple proof. +- Chosen baseline: preserve current-main v42 before v44 activation. +- Chosen downgrade behavior: reject after monotonic epoch commit. +- Chosen direct-ingress protection: local DDL gate in addition to proxy admission. +- Non-blocking follow-up: add dedicated activation/fan-out metrics and publish N-CN staging latency evidence. +- Design gate: revision 1 was explicitly approved on 2026-08-30; no blocking design item remains. diff --git a/pkg/clusterservice/cluster.go b/pkg/clusterservice/cluster.go index 2b19e767f2906..eb51088f3f448 100644 --- a/pkg/clusterservice/cluster.go +++ b/pkg/clusterservice/cluster.go @@ -655,6 +655,8 @@ func newCNService(cn logpb.CNStore) metadata.CNService { ViewMetadataAdmissionGeneration: cn.ViewMetadataAdmissionGeneration, ViewMetadataAdmissionReady: cn.ViewMetadataAdmissionReady, ViewMetadataObservedEpoch: cn.ViewMetadataObservedEpoch, + DDLVisibilityBarrierReady: cn.DDLVisibilityBarrierReady, + ViewMetadataIngressReady: cn.ViewMetadataIngressReady, } } diff --git a/pkg/clusterservice/cluster_test.go b/pkg/clusterservice/cluster_test.go index 49b488c0aa81c..36f1fd8772f22 100644 --- a/pkg/clusterservice/cluster_test.go +++ b/pkg/clusterservice/cluster_test.go @@ -169,6 +169,8 @@ func TestClusterAdmissionSnapshotFiltersEveryPublicInventory(t *testing.T) { UUID: "pending", WorkState: metadata.WorkState_Working, ViewMetadataAdmissionGeneration: 11, + DDLVisibilityBarrierReady: true, + ViewMetadataIngressReady: true, }, } hc.Unlock() @@ -189,12 +191,20 @@ func TestClusterAdmissionSnapshotFiltersEveryPublicInventory(t *testing.T) { require.Equal(t, []string{"ready"}, withoutWorkState) var raw []string + pendingBarrierReady := false + pendingIngressReady := false require.NoError(t, GetCNServiceRawWithContext( context.Background(), c, NewSelector(), func(service metadata.CNService) bool { raw = append(raw, service.ServiceID) + if service.ServiceID == "pending" { + pendingBarrierReady = service.DDLVisibilityBarrierReady + pendingIngressReady = service.ViewMetadataIngressReady + } return true })) require.ElementsMatch(t, []string{"ready", "pending"}, raw) + require.True(t, pendingBarrierReady) + require.True(t, pendingIngressReady) admission := c.GetViewMetadataAdmission() require.True(t, admission.Enabled) diff --git a/pkg/cnservice/server.go b/pkg/cnservice/server.go index 27dc75faa27b6..1016f1e9cb8b5 100644 --- a/pkg/cnservice/server.go +++ b/pkg/cnservice/server.go @@ -155,15 +155,19 @@ func NewService( UUID: cfg.UUID, Role: metadata.MustParseCNRole(cfg.Role), }, - cfg: cfg, - logger: logutil.GetGlobalLogger().Named("cn-service"), - metadataFS: metadataFS, - etlFS: etlFS, - fileService: fileService, - sessionMgr: queryservice.NewSessionManager(), - addressMgr: address.NewAddressManager(cfg.ServiceHost, cfg.PortBase), - gossipNode: gossipNode, - } + cfg: cfg, + logger: logutil.GetGlobalLogger().Named("cn-service"), + metadataFS: metadataFS, + etlFS: etlFS, + fileService: fileService, + sessionMgr: queryservice.NewSessionManager(), + addressMgr: address.NewAddressManager(cfg.ServiceHost, cfg.PortBase), + gossipNode: gossipNode, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + srv.ddlCommitGate.SetFrontierPublisher(srv.publishDDLCommitFrontier) + runtime.ServiceRuntime(cfg.UUID).SetGlobalVariables( + frontend.DDLCommitGateRuntimeKey, srv.ddlCommitGate) srv.colexecServer = colexec.NewServer(cfg.UUID) srv.requestHandler = func(ctx context.Context, @@ -463,6 +467,9 @@ func (s *service) Start() (err error) { if err = s.startUnlessViewMetadataGenerationRevoked(s.queryService.Start); err != nil { return err } + if err = s.prepareDDLVisibilityBarrier(); err != nil { + return err + } if err = s.startFrontendUnlessViewMetadataGenerationRevoked(); err != nil { return err } @@ -477,11 +484,9 @@ func (s *service) Start() (err error) { if err = s.checkViewMetadataGenerationRevoked(); err != nil { return err } - s.viewMetadataIngressReady.Store(true) - if err = s.checkViewMetadataGenerationRevoked(); err != nil { + if err = s.publishDDLVisibilityIngressAfterStart(); err != nil { return err } - s.notifyHeartbeat() if err = s.checkViewMetadataGenerationRevoked(); err != nil { return err @@ -508,12 +513,18 @@ func (s *service) closeService() error { defer logutil.LogClose(s.logger, "cnservice")() s.closeViewMetadataAdmission() - // Stop waits for any in-flight periodic heartbeat before teardown. Keep - // ingress published until all local entry points and work have drained; - // withdrawal below is the ownership handoff linearization point. + // Seal DDL admission before stopping periodic heartbeats. DDL barrier + // withdrawal happens while QueryService remains available for stale-target + // retries; view-admission ownership is withdrawn later, after all local + // ingress and TaskRunner work have drained. + s.ddlVisibilityBarrierClosing.Store(true) + if s.ddlCommitGate != nil { + s.ddlCommitGate.Close() + } s.stopper.Stop() s.closeErr = closeCNServiceSteps( + s.withdrawDDLVisibilityBarrier, // Query commands can reach frontend, task, engine, lock, shard, // auto-increment, and transaction state. Stop and drain this remote // ingress before clearing any of those dependencies. diff --git a/pkg/cnservice/server_ddl_visibility.go b/pkg/cnservice/server_ddl_visibility.go new file mode 100644 index 0000000000000..de69099773a0b --- /dev/null +++ b/pkg/cnservice/server_ddl_visibility.go @@ -0,0 +1,783 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package cnservice + +import ( + "context" + "errors" + "fmt" + "time" + + "github.com/matrixorigin/matrixone/pkg/clusterservice" + "github.com/matrixorigin/matrixone/pkg/common/moerr" + moruntime "github.com/matrixorigin/matrixone/pkg/common/runtime" + "github.com/matrixorigin/matrixone/pkg/defines" + logservicepb "github.com/matrixorigin/matrixone/pkg/pb/logservice" + "github.com/matrixorigin/matrixone/pkg/pb/metadata" + "github.com/matrixorigin/matrixone/pkg/util/file" + "github.com/matrixorigin/matrixone/pkg/util/protoc" +) + +func ddlVisibilityBarrierSupported(serviceID string) bool { + value, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + version, valid := value.(int64) + return ok && valid && version >= defines.MORPCVersion44 +} + +// prepareDDLVisibilityBarrier publishes this CN only after QueryService is +// listening. With protocol v43 active, it then catches up to the largest +// frontier held by the already-published barrier participants before public +// SQL ingress can be admitted. +func (s *service) prepareDDLVisibilityBarrier() error { + // MORPCLatestVersion describes compiled capability, not deployment-wide + // activation. Restore the durable per-CN deployed protocol before deciding + // whether this restart can produce v43 DDL. A fresh upgraded process has no + // marker and preserves the already-deployed v42 baseline until the + // complete-target cut persists v43. + deployedVersion := s.loadDDLVisibilityDeployedProtocol() + if deployedVersion == 0 && s._hakeeperClient != nil { + ctx, cancel := s.newDDLVisibilityBarrierContext(context.Background()) + details, err := s._hakeeperClient.GetClusterDetails(ctx) + if err != nil { + err = moerr.AttachCause(ctx, err) + cancel() + return err + } + cancel() + if details.DDLVisibilityDeployedProtocol >= defines.MORPCVersion44 { + // Markerless scale-out/replacement after the cluster cut joins v43 as + // provisional. It can synchronize and receive activation RPCs, but it + // cannot publish ingress or produce DDL before joining the exact cut. + deployedVersion = -details.DDLVisibilityDeployedProtocol + } + } + rt := moruntime.ServiceRuntime(s.cfg.UUID) + if deployedVersion >= defines.MORPCVersion44 { + s.ddlVisibilityActivationComplete.Store(true) + rt.SetGlobalVariables(moruntime.MOProtocolVersion, deployedVersion) + } else if deployedVersion <= -defines.MORPCVersion44 { + // A provisional marker proves this CN durably fenced its local producers, + // but not that every target committed the same cut. Keep v43 capability so + // retries can converge while startup/public ingress remains fail-closed. + rt.SetGlobalVariables(moruntime.MOProtocolVersion, -deployedVersion) + } else if value, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion); ok { + if version, valid := value.(int64); valid && version >= defines.MORPCVersion44 { + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion42) + } + } + + s.ddlVisibilityBarrierMu.Lock() + defer s.ddlVisibilityBarrierMu.Unlock() + if err := s.prepareDDLVisibilityBarrierLocked(); err != nil { + return err + } + s.ddlVisibilityBarrierPrepared.Store(true) + return nil +} + +func (s *service) prepareDDLVisibilityBarrierLocked() error { + supported := ddlVisibilityBarrierSupported(s.cfg.UUID) + if supported && (s.moCluster == nil || s.queryClient == nil || s._txnClient == nil || + s._hakeeperClient == nil || s.config == nil) { + // Focused service tests may construct only the dependencies relevant to + // their lifecycle assertion. Production NewService initializes a non-zero + // admission generation together with all three barrier dependencies. + if s.viewMetadataAdmissionGeneration != 0 { + return moerr.NewInternalErrorNoCtx("DDL visibility barrier dependencies are unavailable") + } + } + + s.ddlVisibilityBarrierReady.Store(true) + if !supported || s.moCluster == nil || s.queryClient == nil || s._txnClient == nil { + s.notifyHeartbeat() + return nil + } + + timeout := s.cfg.HAKeeper.DiscoveryTimeout.Duration + if timeout <= 0 { + timeout = 30 * time.Second + } + ctx, cancel := context.WithTimeout(context.Background(), timeout) + defer cancel() + + s.ddlCommitGate.RecordDDLFrontier(s._txnClient.GetLatestCommitTS()) + if s.ddlVisibilityActivationComplete.Load() { + // No DDL producer exists before listeners start, so this new incarnation + // is already drained and may publish its own Prepared proof. + s.ddlVisibilityActivationPrepared.Store(true) + } + // Startup owns this mutex, so relying on the periodic heartbeat (which uses + // the same mutex to order readiness snapshots) would deadlock publication. + // Publish the startup barrier directly before waiting for its authoritative + // observation; periodic heartbeats remain serialized behind startup. + if _, err := s.sendCNHeartbeat(ctx, s.newCNStoreHeartbeat()); err != nil { + return moerr.AttachCause(ctx, err) + } + + retryInterval := s.cfg.HAKeeper.HeatbeatInterval.Duration + if retryInterval < minClusterReadinessRetryInterval { + retryInterval = minClusterReadinessRetryInterval + } + if err := s.waitForDDLVisibilityBarrierPublication(ctx, retryInterval); err != nil { + return err + } + if err := s.syncStartupDDLVisibilityFrontier(ctx); err != nil { + return err + } + if s.ddlVisibilityActivationComplete.Load() { + // The reconstructed frontier is now applied locally. Publish a Fenced + // proof bound to this restart/replacement incarnation so future scale-out + // activation does not wait on stale proof from its predecessor. + s.ddlVisibilityActivationFenced.Store(true) + if _, err := s.sendCNHeartbeat(ctx, s.newCNStoreHeartbeat()); err != nil { + return moerr.AttachCause(ctx, err) + } + } + return nil +} + +// withdrawDDLVisibilityBarrier closes both externally published gates before +// QueryService is stopped. closeService invokes it only after the periodic +// heartbeat task has terminated, so no previously captured ready heartbeat can +// overwrite this final withdrawal. +func (s *service) publishDDLVisibilityIngressAfterStart() error { + // Serialize listener-ready publication with startup/activation/shutdown. + // Compiled v43 capability is not evidence that the deployment-wide producer + // cut completed: a default-v43 CN stays fail-closed until its complete-target + // activation succeeds. If activation raced ahead of listener startup, this + // method remains the sole owner that opens ingress after listeners are live. + s.ddlVisibilityBarrierMu.Lock() + defer s.ddlVisibilityBarrierMu.Unlock() + if err := s.checkViewMetadataGenerationRevoked(); err != nil { + return err + } + s.ddlVisibilityListenersReady.Store(true) + if ddlVisibilityBarrierSupported(s.cfg.UUID) && + !s.ddlVisibilityActivationComplete.Load() { + s.viewMetadataIngressReady.Store(false) + s.notifyHeartbeat() + return nil + } + if !ddlVisibilityBarrierSupported(s.cfg.UUID) && s._hakeeperClient != nil { + // Atomically ask HAKeeper to publish this markerless CN as ingress-ready. + // The RSM either records this join before the cut (so activation must + // include it), or rejects it after the committed epoch and returns that + // epoch in the same transition. A separate epoch read is a TOCTOU gap. + hb := s.newCNStoreHeartbeat() + hb.ViewMetadataIngressReady = true + ctx, cancel := s.newDDLVisibilityBarrierContext(context.Background()) + batch, err := s.sendCNHeartbeat(ctx, hb) + if err != nil { + err = moerr.AttachCause(ctx, err) + cancel() + return err + } + cancel() + if batch.DDLVisibilityDeployedProtocol >= defines.MORPCVersion44 { + moruntime.ServiceRuntime(s.cfg.UUID).SetGlobalVariables( + moruntime.MOProtocolVersion, batch.DDLVisibilityDeployedProtocol) + s.viewMetadataIngressReady.Store(false) + return nil + } + } + if s.ddlCommitGate != nil { + s.ddlCommitGate.EnablePublicDDL() + } + s.viewMetadataIngressReady.Store(true) + if err := s.checkViewMetadataGenerationRevoked(); err != nil { + return err + } + s.notifyHeartbeat() + return nil +} + +func (s *service) withdrawDDLVisibilityBarrier() error { + s.ddlVisibilityBarrierMu.Lock() + defer s.ddlVisibilityBarrierMu.Unlock() + ctx, cancel := s.newDDLVisibilityBarrierContext(context.Background()) + defer cancel() + return s.withdrawDDLVisibilityBarrierLocked(ctx) +} + +func (s *service) withdrawDDLVisibilityBarrierLocked(ctx context.Context) error { + s.ddlVisibilityHeartbeatMu.Lock() + defer s.ddlVisibilityHeartbeatMu.Unlock() + s.viewMetadataIngressReady.Store(false) + s.ddlVisibilityBarrierReady.Store(false) + if s.viewMetadataAdmissionGeneration == 0 { + return nil + } + if s.cfg == nil || s._hakeeperClient == nil || s.moCluster == nil || s.config == nil { + return moerr.NewInternalErrorNoCtx("DDL visibility barrier withdrawal dependencies are unavailable") + } + + if _, err := s.sendCNHeartbeat(ctx, s.newCNStoreHeartbeat()); err != nil { + return moerr.AttachCause(ctx, err) + } + return s.waitForDDLVisibilityBarrierWithdrawal(ctx, s.ddlVisibilityBarrierRetryInterval()) +} + +func (s *service) newDDLVisibilityBarrierContext(parent context.Context) (context.Context, context.CancelFunc) { + timeout := 30 * time.Second + if s.cfg != nil && s.cfg.HAKeeper.DiscoveryTimeout.Duration > 0 { + timeout = s.cfg.HAKeeper.DiscoveryTimeout.Duration + } + return context.WithTimeout(parent, timeout) +} + +func (s *service) ddlVisibilityBarrierRetryInterval() time.Duration { + retryInterval := time.Duration(0) + if s.cfg != nil { + retryInterval = s.cfg.HAKeeper.HeatbeatInterval.Duration + } + if retryInterval < minClusterReadinessRetryInterval { + retryInterval = minClusterReadinessRetryInterval + } + return retryInterval +} + +// setProtocolVersion serializes live activation with startup and shutdown. +// Every target CN blocks and drains local DDL before publishing Prepared. Once +// all targets are prepared, no v34 DDL producer exists; each CN applies the +// converged frontier and publishes Fenced. A CN releases its DDL gate only after +// every target is fenced, at which point all later DDL uses the v43 fan-out and +// every still-fencing receiver remains barrier-reachable. +func (s *service) setProtocolVersion(ctx context.Context, version int64, targets []string) error { + s.ddlVisibilityBarrierMu.Lock() + defer s.ddlVisibilityBarrierMu.Unlock() + + if s.cfg == nil { + return moerr.NewInternalError(ctx, "CN configuration is unavailable") + } + rt := moruntime.ServiceRuntime(s.cfg.UUID) + value, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + if !ok { + return moerr.NewInternalError(ctx, "protocol version not found") + } + current, ok := value.(int64) + if !ok { + return moerr.NewInternalError(ctx, "invalid protocol version") + } + pending := s.ddlVisibilityActivationPending.Load() + if version < defines.MORPCVersion44 { + if pending { + return moerr.NewInvalidStateNoCtx("cannot downgrade during DDL visibility activation") + } + deployed := s.loadDDLVisibilityDeployedProtocol() + clusterCommitted := s.ddlVisibilityActivationComplete.Load() || + deployed >= defines.MORPCVersion44 || deployed <= -defines.MORPCVersion44 + if !clusterCommitted && s._hakeeperClient != nil { + queryCtx, cancel := s.newDDLVisibilityBarrierContext(ctx) + details, err := s._hakeeperClient.GetClusterDetails(queryCtx) + if err != nil { + err = moerr.AttachCause(queryCtx, err) + cancel() + return err + } + cancel() + clusterCommitted = details.DDLVisibilityDeployedProtocol >= defines.MORPCVersion44 + } + if clusterCommitted { + return moerr.NewInvalidStateNoCtx( + "cannot downgrade after the DDL visibility cluster epoch is committed") + } + if err := s.persistDDLVisibilityDeployedProtocol(version); err != nil { + return err + } + rt.SetGlobalVariables(moruntime.MOProtocolVersion, version) + s.ddlVisibilityActivationPrepared.Store(false) + s.ddlVisibilityActivationFenced.Store(false) + s.ddlVisibilityActivationComplete.Store(false) + return nil + } + if !s.ddlVisibilityBarrierPrepared.Load() { + rt.SetGlobalVariables(moruntime.MOProtocolVersion, version) + return nil + } + if current >= defines.MORPCVersion44 && !pending && + s.ddlVisibilityActivationComplete.Load() { + rt.SetGlobalVariables(moruntime.MOProtocolVersion, version) + return nil + } + if s.ddlVisibilityBarrierClosing.Load() || s.viewMetadataGenerationRevoked.Load() { + return moerr.NewServiceUnavailableNoCtx("CN is closing") + } + if s.viewMetadataAdmissionGeneration == 0 || s._hakeeperClient == nil || + s.moCluster == nil || s.queryClient == nil || s._txnClient == nil || s.config == nil || + s.ddlCommitGate == nil { + return moerr.NewInternalError(ctx, "DDL visibility activation dependencies are unavailable") + } + activationTargets, err := validateDDLVisibilityActivationTargets(s.cfg.UUID, targets) + if err != nil { + return err + } + if err := s.requireDDLVisibilityEpochHAKeeperCapability(ctx); err != nil { + return err + } + + barrierCtx, cancel := s.newDDLVisibilityBarrierContext(ctx) + defer cancel() + if !pending { + // A default-v43 process deliberately keeps ingress closed until the + // complete-target cut. Restore ingress when listeners are already live; + // if activation races before listener startup, Start opens it afterward. + s.ddlVisibilityRestoreIngress.Store( + s.viewMetadataIngressReady.Load() || s.ddlVisibilityListenersReady.Load()) + } + s.ddlVisibilityActivationPending.Store(true) + s.ddlVisibilityActivationPrepared.Store(false) + s.ddlVisibilityActivationFenced.Store(false) + // Stop new old-protocol DDL admission before the fallible authoritative + // withdrawal. Both withdrawal and subsequent drain failures deliberately + // leave the gate blocked for a fail-closed retry. + if err := s.ddlCommitGate.BlockNew(); err != nil { + return err + } + if err := s.setDDLVisibilityIngressLocked(barrierCtx, false); err != nil { + return err + } + if err := s.ddlCommitGate.WaitDrained(barrierCtx); err != nil { + return err + } + // Capture a durable committed high-water mark after every local DDL producer + // is drained. A restarted incarnation's txn client reconstructs a frontier + // at or beyond DDL committed by its predecessor. + s.ddlCommitGate.RecordDDLFrontier(s._txnClient.GetLatestCommitTS()) + + // Version 43 is visible only after the local pre-v43 producers are drained. It + // acts as the distributed Prepared signal queried by every other target. + rt.SetGlobalVariables(moruntime.MOProtocolVersion, version) + s.ddlVisibilityActivationPrepared.Store(true) + if err := s.publishDDLVisibilityActivationPhaseLocked(barrierCtx); err != nil { + return err + } + if err := s.waitForDDLVisibilityActivationPhase( + barrierCtx, activationTargets, false); err != nil { + return err + } + if err := s.syncStartupDDLVisibilityFrontier(barrierCtx); err != nil { + return err + } + // Fenced is a distributed proof. Publish it only after the local provisional + // state is durable, so a persistence failure can never be observed as Fence. + if err := s.persistDDLVisibilityDeployedProtocol(-version); err != nil { + return err + } + s.ddlVisibilityActivationFenced.Store(true) + if err := s.publishDDLVisibilityActivationPhaseLocked(barrierCtx); err != nil { + return err + } + if err := s.waitForDDLVisibilityActivationPhase( + barrierCtx, activationTargets, true); err != nil { + return err + } + if s.ddlVisibilityBarrierClosing.Load() || s.viewMetadataGenerationRevoked.Load() { + return moerr.NewServiceUnavailableNoCtx("CN is closing") + } + // Re-read authoritative membership after the final phase scan. A markerless + // CN may have joined after dispatch; committing without it would reopen an + // un-fenced legacy producer. The join either appears here and aborts this cut, + // or its atomic ingress heartbeat observes the already committed epoch. + commitTargets, err := s.revalidateDDLVisibilityActivationMembership(barrierCtx, activationTargets) + if err != nil { + return err + } + if err := s.commitDDLVisibilityClusterEpoch(barrierCtx, version, commitTargets); err != nil { + return err + } + // Commit the deployed cut before reopening ingress. A failure leaves the + // durable provisional marker in place; restart therefore remains on v43 but + // fail-closed until a complete-target retry commits the cut. + if err := s.persistDDLVisibilityDeployedProtocol(version); err != nil { + return err + } + if err := s.setDDLVisibilityIngressLocked( + barrierCtx, s.ddlVisibilityRestoreIngress.Load()); err != nil { + cleanupCtx, cleanupCancel := s.newDDLVisibilityBarrierContext(context.Background()) + cleanupErr := s.setDDLVisibilityIngressLocked(cleanupCtx, false) + cleanupCancel() + return errors.Join(err, cleanupErr) + } + + s.ddlVisibilityActivationComplete.Store(true) + s.ddlVisibilityActivationPending.Store(false) + if s.ddlVisibilityRestoreIngress.Load() { + s.ddlCommitGate.EnablePublicDDL() + } + s.ddlCommitGate.Unblock() + return nil +} + +func (s *service) loadDDLVisibilityDeployedProtocol() int64 { + return s.metadata.DDLVisibilityDeployedProtocol +} + +func (s *service) persistDDLVisibilityDeployedProtocol(version int64) error { + if s.metadataFS == nil { + // Focused service tests may omit the local metadata file service, but the + // process-local state must still model the committed heartbeat that follows. + s.metadata.DDLVisibilityDeployedProtocol = version + return nil + } + previous := s.metadata.DDLVisibilityDeployedProtocol + s.metadata.DDLVisibilityDeployedProtocol = version + if err := file.WriteFile(s.metadataFS, getMetadataFile(s.cfg.UUID), protoc.MustMarshal(&s.metadata)); err != nil { + s.metadata.DDLVisibilityDeployedProtocol = previous + return err + } + return nil +} + +func (s *service) revalidateDDLVisibilityActivationMembership( + ctx context.Context, + targets map[string]struct{}, +) ([]logservicepb.DDLVisibilityActivationTarget, error) { + details, err := s._hakeeperClient.GetClusterDetails(ctx) + if err != nil { + return nil, moerr.AttachCause(ctx, err) + } + commitTargets := make([]logservicepb.DDLVisibilityActivationTarget, 0, len(targets)) + seen := 0 + for _, cn := range details.CNStores { + if cn.QueryAddress == "" || cn.ViewMetadataAdmissionGeneration == 0 { + continue + } + seen++ + if _, ok := targets[cn.UUID]; !ok { + return nil, moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "DDL visibility activation membership changed: authoritative CN %s is not fenced", cn.UUID)) + } + if !cn.DDLVisibilityBarrierReady { + return nil, moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "authoritative CN %s does not support the DDL visibility activation receiver", cn.UUID)) + } + commitTargets = append(commitTargets, logservicepb.DDLVisibilityActivationTarget{ + ServiceID: cn.UUID, Generation: cn.ViewMetadataAdmissionGeneration, QueryAddress: cn.QueryAddress, + }) + } + if seen != len(targets) { + return nil, moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "DDL visibility activation membership changed: targets=%d authoritative=%d", len(targets), seen)) + } + return commitTargets, nil +} + +func (s *service) commitDDLVisibilityClusterEpoch( + ctx context.Context, + version int64, + targets []logservicepb.DDLVisibilityActivationTarget, +) error { + hb := s.newCNStoreHeartbeat() + hb.DDLVisibilityDeployedProtocol = version + hb.DDLVisibilityEpochCommitTargets = append( + []logservicepb.DDLVisibilityActivationTarget(nil), targets...) + batch, err := s.sendCNHeartbeat(ctx, hb) + if err != nil { + return moerr.AttachCause(ctx, err) + } + if batch.DDLVisibilityDeployedProtocol < version { + return moerr.NewInvalidStateNoCtx( + "DDL visibility activation membership changed before atomic cluster epoch commit") + } + return nil +} + +func (s *service) requireDDLVisibilityEpochHAKeeperCapability(ctx context.Context) error { + queryCtx, cancel := s.newDDLVisibilityBarrierContext(ctx) + defer cancel() + details, err := s._hakeeperClient.GetClusterDetails(queryCtx) + if err != nil { + return moerr.AttachCause(queryCtx, err) + } + if len(details.LogStores) == 0 { + return moerr.NewInvalidStateNoCtx("DDL visibility activation requires HAKeeper capability inventory") + } + for _, store := range details.LogStores { + if !store.DDLVisibilityDeployedProtocolSupported { + return moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "LogStore %s does not support the durable DDL visibility deployment epoch", store.UUID)) + } + } + return nil +} + +func validateDDLVisibilityActivationTargets(serviceID string, targets []string) (map[string]struct{}, error) { + result := make(map[string]struct{}, len(targets)) + for _, target := range targets { + if target == "" { + return nil, moerr.NewInternalErrorNoCtx("DDL visibility activation target is empty") + } + if _, exists := result[target]; exists { + return nil, moerr.NewInternalErrorNoCtxf( + "DDL visibility activation target %s is duplicated", target) + } + result[target] = struct{}{} + } + if _, ok := result[serviceID]; !ok { + return nil, moerr.NewInternalErrorNoCtxf( + "DDL visibility activation targets do not include local CN %s", serviceID) + } + return result, nil +} + +func (s *service) setDDLVisibilityIngressLocked(ctx context.Context, ready bool) error { + s.ddlVisibilityHeartbeatMu.Lock() + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(ready) + _, err := s.sendCNHeartbeat(ctx, s.newCNStoreHeartbeat()) + s.ddlVisibilityHeartbeatMu.Unlock() + if err != nil { + return moerr.AttachCause(ctx, err) + } + return s.waitForDDLVisibilityIngress( + ctx, s.ddlVisibilityBarrierRetryInterval(), ready) +} + +func (s *service) waitForDDLVisibilityIngress( + ctx context.Context, + retryInterval time.Duration, + ready bool, +) error { + refresher, ok := s.moCluster.(clusterservice.AuthoritativeRefresher) + if !ok { + return moerr.NewInternalErrorNoCtx( + "CN cluster service does not support authoritative DDL visibility refresh") + } + for { + if err := refresher.Refresh(ctx); err == nil { + matched := false + err = clusterservice.GetCNServiceRawWithContext( + ctx, + s.moCluster, + clusterservice.NewServiceIDSelector(s.cfg.UUID), + func(cn metadata.CNService) bool { + matched = cn.ViewMetadataAdmissionGeneration == s.viewMetadataAdmissionGeneration && + cn.DDLVisibilityBarrierReady && cn.ViewMetadataIngressReady == ready + return false + }) + if err != nil { + return err + } + if matched { + return nil + } + } + if err := waitDDLVisibilityRetry(ctx, retryInterval); err != nil { + return moerr.NewInternalErrorf( + context.Background(), + "CN %s DDL visibility ingress=%t was not published before deadline: %v", + s.cfg.UUID, + ready, + err) + } + } +} + +func (s *service) publishDDLVisibilityActivationPhaseLocked(ctx context.Context) error { + s.ddlVisibilityHeartbeatMu.Lock() + defer s.ddlVisibilityHeartbeatMu.Unlock() + _, err := s.sendCNHeartbeat(ctx, s.newCNStoreHeartbeat()) + if err != nil { + return moerr.AttachCause(ctx, err) + } + return nil +} + +func (s *service) waitForDDLVisibilityActivationPhase( + ctx context.Context, + targets map[string]struct{}, + requireFenced bool, +) error { + if s._hakeeperClient == nil { + return moerr.NewInternalErrorNoCtx("HAKeeper client is unavailable during DDL visibility activation") + } + var lastErr error + for { + allReady := true + seen := make(map[string]struct{}, len(targets)) + details, err := s._hakeeperClient.GetClusterDetails(ctx) + if err != nil { + lastErr = err + allReady = false + } else { + for _, cn := range details.CNStores { + _, expected := targets[cn.UUID] + if cn.QueryAddress == "" || cn.ViewMetadataAdmissionGeneration == 0 { + if expected { + return moerr.NewInvalidStateNoCtxf( + "DDL visibility activation target %s has no authoritative identity", cn.UUID) + } + continue + } + if !expected { + return moerr.NewInvalidStateNoCtxf( + "DDL visibility activation target set omits authoritative CN %s", cn.UUID) + } + seen[cn.UUID] = struct{}{} + if !cn.DDLVisibilityActivationPrepared || + (requireFenced && !cn.DDLVisibilityActivationFenced) { + allReady = false + } + } + } + if len(seen) != len(targets) { + allReady = false + } + if allReady { + return nil + } + if err := waitDDLVisibilityRetry(ctx, s.ddlVisibilityBarrierRetryInterval()); err != nil { + return moerr.NewInternalErrorf( + context.Background(), + "DDL visibility activation fenced=%t did not converge before deadline: %v (last error: %v)", + requireFenced, + err, + lastErr) + } + } +} + +func waitDDLVisibilityRetry(ctx context.Context, retryInterval time.Duration) error { + timer := time.NewTimer(retryInterval) + defer timer.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-timer.C: + return nil + } +} + +func (s *service) waitForDDLVisibilityBarrierWithdrawal( + ctx context.Context, + retryInterval time.Duration, +) error { + refresher, ok := s.moCluster.(clusterservice.AuthoritativeRefresher) + if !ok { + return moerr.NewInternalErrorNoCtx( + "CN cluster service does not support authoritative DDL visibility refresh") + } + + for { + if err := refresher.Refresh(ctx); err == nil { + withdrawn := true + err = clusterservice.GetCNServiceRawWithContext( + ctx, + s.moCluster, + clusterservice.NewServiceIDSelector(s.cfg.UUID), + func(cn metadata.CNService) bool { + if cn.ViewMetadataAdmissionGeneration < s.viewMetadataAdmissionGeneration || + (cn.ViewMetadataAdmissionGeneration == s.viewMetadataAdmissionGeneration && + cn.DDLVisibilityBarrierReady) { + withdrawn = false + } + return false + }) + if err != nil { + return err + } + if withdrawn { + return nil + } + } + + timer := time.NewTimer(retryInterval) + select { + case <-ctx.Done(): + if !timer.Stop() { + select { + case <-timer.C: + default: + } + } + return moerr.NewInternalErrorf( + context.Background(), + "CN %s DDL visibility barrier was not withdrawn before shutdown deadline: %v", + s.cfg.UUID, + ctx.Err()) + case <-timer.C: + } + } +} + +func (s *service) waitForDDLVisibilityBarrierPublication( + ctx context.Context, + retryInterval time.Duration, +) error { + refresher, ok := s.moCluster.(clusterservice.AuthoritativeRefresher) + if !ok { + return moerr.NewInternalErrorNoCtx( + "CN cluster service does not support authoritative DDL visibility refresh") + } + + for { + if err := refresher.Refresh(ctx); err == nil { + published := false + err = clusterservice.GetCNServiceRawWithContext( + ctx, + s.moCluster, + clusterservice.NewServiceIDSelector(s.cfg.UUID), + func(cn metadata.CNService) bool { + published = cn.ViewMetadataAdmissionGeneration == s.viewMetadataAdmissionGeneration && + cn.DDLVisibilityBarrierReady + return false + }) + if err != nil { + return err + } + if published { + return nil + } + } + + timer := time.NewTimer(retryInterval) + select { + case <-ctx.Done(): + if !timer.Stop() { + select { + case <-timer.C: + default: + } + } + return moerr.NewInternalErrorf( + context.Background(), + "CN %s DDL visibility barrier was not published before startup deadline: %v", + s.cfg.UUID, + ctx.Err()) + case <-timer.C: + } + } +} + +func (s *service) syncStartupDDLVisibilityFrontier(ctx context.Context) error { + if s._hakeeperClient == nil { + return moerr.NewInternalErrorNoCtx("HAKeeper client is unavailable during DDL visibility frontier sync") + } + details, err := s._hakeeperClient.GetClusterDetails(ctx) + if err != nil { + return moerr.AttachCause(ctx, err) + } + // HAKeeper retains this maximum independently of current CN membership, so + // producer restart/replacement/removal cannot erase a committed DDL fact. + maxTS := details.DDLVisibilityFrontier + if maxTS.IsEmpty() { + return nil + } + if _, err := s._txnClient.WaitLogTailAppliedAt(ctx, maxTS); err != nil { + return err + } + s._txnClient.SyncLatestCommitTS(maxTS) + return nil +} diff --git a/pkg/cnservice/server_ddl_visibility_test.go b/pkg/cnservice/server_ddl_visibility_test.go new file mode 100644 index 0000000000000..ff7ecbbaaa8a2 --- /dev/null +++ b/pkg/cnservice/server_ddl_visibility_test.go @@ -0,0 +1,1803 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package cnservice + +import ( + "context" + "errors" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/golang/mock/gomock" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/matrixorigin/matrixone/pkg/clusterservice" + "github.com/matrixorigin/matrixone/pkg/common/morpc" + moruntime "github.com/matrixorigin/matrixone/pkg/common/runtime" + "github.com/matrixorigin/matrixone/pkg/common/stopper" + "github.com/matrixorigin/matrixone/pkg/defines" + "github.com/matrixorigin/matrixone/pkg/fileservice" + "github.com/matrixorigin/matrixone/pkg/frontend" + mock_frontend "github.com/matrixorigin/matrixone/pkg/frontend/test" + "github.com/matrixorigin/matrixone/pkg/gossip" + "github.com/matrixorigin/matrixone/pkg/lockservice" + "github.com/matrixorigin/matrixone/pkg/logservice" + logservicepb "github.com/matrixorigin/matrixone/pkg/pb/logservice" + "github.com/matrixorigin/matrixone/pkg/pb/metadata" + "github.com/matrixorigin/matrixone/pkg/pb/query" + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" + "github.com/matrixorigin/matrixone/pkg/util" +) + +type failReplaceMetadataFS struct { + fileservice.ReplaceableFileService + failAt int + calls int +} + +func (f *failReplaceMetadataFS) Replace(ctx context.Context, vector fileservice.IOVector) error { + f.calls++ + if f.calls == f.failAt { + return errors.New("injected metadata replace failure") + } + return f.ReplaceableFileService.Replace(ctx, vector) +} + +func newDDLVisibilityMetadataFS(t *testing.T) fileservice.ReplaceableFileService { + t.Helper() + fs, err := fileservice.NewMemoryFS(defines.LocalFileServiceName, fileservice.DisabledCacheConfig, nil) + require.NoError(t, err) + return fs +} + +type ddlVisibilityTestCluster struct { + cnServices []metadata.CNService + refreshCalls int + refreshHook func() + phaseMu sync.Mutex + phases map[string]query.GetProtocolVersionResponse + frontiers map[string]timestamp.Timestamp + globalFrontier timestamp.Timestamp + phaseHook func(string) query.GetProtocolVersionResponse +} + +func (c *ddlVisibilityTestCluster) GetCNService( + selector clusterservice.Selector, + apply func(metadata.CNService) bool, +) { + c.forEachCN(selector, apply) +} + +func (*ddlVisibilityTestCluster) GetTNService( + clusterservice.Selector, + func(metadata.TNService) bool, +) { +} + +func (*ddlVisibilityTestCluster) GetAllTNServices() []metadata.TNService { return nil } + +func (c *ddlVisibilityTestCluster) GetCNServiceWithoutWorkingState( + selector clusterservice.Selector, + apply func(metadata.CNService) bool, +) { + c.forEachCN(selector, apply) +} + +func (c *ddlVisibilityTestCluster) forEachCN( + selector clusterservice.Selector, + apply func(metadata.CNService) bool, +) { + for _, cn := range c.cnServices { + if selector.MatchCN(cn) && !apply(cn) { + return + } + } +} + +func (*ddlVisibilityTestCluster) ForceRefresh(bool) {} +func (c *ddlVisibilityTestCluster) Refresh(context.Context) error { + c.refreshCalls++ + if c.refreshHook != nil { + c.refreshHook() + } + return nil +} +func (*ddlVisibilityTestCluster) Close() {} +func (*ddlVisibilityTestCluster) DebugUpdateCNLabel(string, map[string][]string) error { + return nil +} +func (*ddlVisibilityTestCluster) DebugUpdateCNWorkState(string, int) error { return nil } +func (*ddlVisibilityTestCluster) RemoveCN(string) {} +func (*ddlVisibilityTestCluster) AddCN(metadata.CNService) {} +func (*ddlVisibilityTestCluster) UpdateCN(metadata.CNService) {} + +type ddlVisibilityTestQueryClient struct { + serviceID string + frontiers map[string]timestamp.Timestamp + protocols map[string]query.GetProtocolVersionResponse + protocolFn func(string) query.GetProtocolVersionResponse + nilProtocol map[string]bool + requests []string + methods []query.CmdMethod + releases int +} + +func (c *ddlVisibilityTestQueryClient) ServiceID() string { return c.serviceID } +func (c *ddlVisibilityTestQueryClient) SendMessage( + _ context.Context, + address string, + req *query.Request, +) (*query.Response, error) { + c.requests = append(c.requests, address) + c.methods = append(c.methods, req.CmdMethod) + if req.CmdMethod == query.CmdMethod_GetProtocolVersion { + if req.GetProtocolVersion == nil { + return nil, errors.New("missing GetProtocolVersion request payload") + } + if c.nilProtocol[address] { + return &query.Response{}, nil + } + protocol := c.protocols[address] + if c.protocolFn != nil { + protocol = c.protocolFn(address) + } + return &query.Response{GetProtocolVersion: &protocol}, nil + } + return &query.Response{GetCommit: &query.GetCommitResponse{ + CurrentCommitTS: c.frontiers[address], + }}, nil +} +func (*ddlVisibilityTestQueryClient) NewRequest(method query.CmdMethod) *query.Request { + return &query.Request{CmdMethod: method} +} +func (c *ddlVisibilityTestQueryClient) Release(*query.Response) { c.releases++ } +func (*ddlVisibilityTestQueryClient) Close() error { return nil } + +func activationTestPeerProtocols() map[string]query.GetProtocolVersionResponse { + return map[string]query.GetProtocolVersionResponse{ + "peer:6001": { + Version: defines.MORPCVersion44, + DDLVisibilityActivationPrepared: true, + DDLVisibilityActivationFenced: true, + }, + } +} + +type ddlVisibilityWithdrawalHAKeeperClient struct { + logservice.CNHAKeeperClient + cluster *ddlVisibilityTestCluster + queryClosed <-chan struct{} + sendErr error + sendErrors map[int]error + heartbeats []logservicepb.CNStoreHeartbeat + queryClosedBeforeSend bool + closeCalls int + clusterDeployedProtocol int64 + oldHAKeeperReplica bool + oldDDLFrontierRSM bool + authoritativeGenerations map[string]uint64 +} + +func (c *ddlVisibilityWithdrawalHAKeeperClient) GetClusterDetails( + context.Context, +) (logservicepb.ClusterDetails, error) { + details := logservicepb.ClusterDetails{ + DDLVisibilityDeployedProtocol: c.clusterDeployedProtocol, + LogStores: []logservicepb.LogStore{{ + UUID: "log-1", DDLVisibilityDeployedProtocolSupported: !c.oldHAKeeperReplica, + }}, + } + if c.cluster != nil { + c.cluster.phaseMu.Lock() + defer c.cluster.phaseMu.Unlock() + details.DDLVisibilityFrontier = c.cluster.globalFrontier + for _, frontier := range c.cluster.frontiers { + if details.DDLVisibilityFrontier.Less(frontier) { + details.DDLVisibilityFrontier = frontier + } + } + for _, cn := range c.cluster.cnServices { + phase, ok := c.cluster.phases[cn.ServiceID] + if c.cluster.phaseHook != nil && len(c.cluster.phases) > 0 { + phase = c.cluster.phaseHook(cn.QueryAddress) + ok = true + } + if !ok { + phase.DDLVisibilityActivationPrepared = cn.DDLVisibilityBarrierReady + phase.DDLVisibilityActivationFenced = cn.DDLVisibilityBarrierReady + } + details.CNStores = append(details.CNStores, logservicepb.CNStore{ + UUID: cn.ServiceID, QueryAddress: cn.QueryAddress, + ViewMetadataAdmissionGeneration: cn.ViewMetadataAdmissionGeneration, + DDLVisibilityBarrierReady: cn.DDLVisibilityBarrierReady, + DDLVisibilityActivationPrepared: phase.DDLVisibilityActivationPrepared, + DDLVisibilityActivationFenced: phase.DDLVisibilityActivationFenced, + DDLVisibilityFrontier: c.cluster.frontiers[cn.ServiceID], + }) + } + } + return details, nil +} + +func (c *ddlVisibilityWithdrawalHAKeeperClient) SendCNHeartbeat( + _ context.Context, + hb logservicepb.CNStoreHeartbeat, +) (logservicepb.CommandBatch, error) { + select { + case <-c.queryClosed: + c.queryClosedBeforeSend = true + default: + } + c.heartbeats = append(c.heartbeats, hb) + if err := c.sendErrors[len(c.heartbeats)]; err != nil { + return logservicepb.CommandBatch{}, err + } + if c.sendErr != nil { + return logservicepb.CommandBatch{}, c.sendErr + } + authoritativeGeneration := hb.ViewMetadataAdmissionGeneration + if generation := c.authoritativeGenerations[hb.UUID]; generation > authoritativeGeneration { + return logservicepb.CommandBatch{ + DDLVisibilityDeployedProtocol: c.clusterDeployedProtocol, + DDLVisibilityFrontier: &c.cluster.globalFrontier, + ViewMetadataAdmission: &logservicepb.ViewMetadataAdmission{ + Generation: generation, + }, + }, nil + } + if c.cluster != nil { + c.cluster.phaseMu.Lock() + if c.cluster.phases == nil { + c.cluster.phases = make(map[string]query.GetProtocolVersionResponse) + } + c.cluster.phases[hb.UUID] = query.GetProtocolVersionResponse{ + Version: hb.DDLVisibilityDeployedProtocol, + DDLVisibilityActivationPrepared: hb.DDLVisibilityActivationPrepared, + DDLVisibilityActivationFenced: hb.DDLVisibilityActivationFenced, + } + if c.cluster.frontiers == nil { + c.cluster.frontiers = make(map[string]timestamp.Timestamp) + } + if !c.oldDDLFrontierRSM { + c.cluster.frontiers[hb.UUID] = hb.DDLVisibilityFrontier + if c.cluster.globalFrontier.Less(hb.DDLVisibilityFrontier) { + c.cluster.globalFrontier = hb.DDLVisibilityFrontier + } + } + c.cluster.phaseMu.Unlock() + } + for i := range c.cluster.cnServices { + cn := &c.cluster.cnServices[i] + if cn.ServiceID == hb.UUID { + cn.ViewMetadataAdmissionGeneration = hb.ViewMetadataAdmissionGeneration + cn.DDLVisibilityBarrierReady = hb.DDLVisibilityBarrierReady + cn.ViewMetadataIngressReady = hb.ViewMetadataIngressReady + if c.clusterDeployedProtocol >= defines.MORPCVersion44 && + hb.DDLVisibilityDeployedProtocol < c.clusterDeployedProtocol { + cn.ViewMetadataIngressReady = false + } + } + } + if len(hb.DDLVisibilityEpochCommitTargets) > 0 { + targets := make(map[string]logservicepb.DDLVisibilityActivationTarget, + len(hb.DDLVisibilityEpochCommitTargets)) + for _, target := range hb.DDLVisibilityEpochCommitTargets { + targets[target.ServiceID] = target + } + matched := 0 + valid := len(targets) == len(hb.DDLVisibilityEpochCommitTargets) + for _, cn := range c.cluster.cnServices { + if cn.QueryAddress == "" || cn.ViewMetadataAdmissionGeneration == 0 { + continue + } + matched++ + target, ok := targets[cn.ServiceID] + valid = valid && ok && target.Generation == cn.ViewMetadataAdmissionGeneration && + target.QueryAddress == cn.QueryAddress && cn.DDLVisibilityBarrierReady + } + if valid && matched == len(targets) { + c.clusterDeployedProtocol = hb.DDLVisibilityDeployedProtocol + } + } + return logservicepb.CommandBatch{ + DDLVisibilityDeployedProtocol: c.clusterDeployedProtocol, + DDLVisibilityFrontier: &c.cluster.globalFrontier, + ViewMetadataAdmission: &logservicepb.ViewMetadataAdmission{ + Generation: authoritativeGeneration, + }, + }, nil +} + +func (c *ddlVisibilityWithdrawalHAKeeperClient) Close() error { + c.closeCalls++ + return nil +} + +type ddlVisibilityHeartbeatOrderClient struct { + logservice.CNHAKeeperClient + mu sync.Mutex + calls int + firstStarted chan struct{} + releaseFirst chan struct{} + completed []logservicepb.CNStoreHeartbeat +} + +func (c *ddlVisibilityHeartbeatOrderClient) SendCNHeartbeat( + _ context.Context, + hb logservicepb.CNStoreHeartbeat, +) (logservicepb.CommandBatch, error) { + c.mu.Lock() + c.calls++ + call := c.calls + c.mu.Unlock() + if call == 1 { + close(c.firstStarted) + <-c.releaseFirst + } + c.mu.Lock() + c.completed = append(c.completed, hb) + c.mu.Unlock() + return logservicepb.CommandBatch{}, errors.New("stop heartbeat after ordering observation") +} + +type ddlVisibilityCloseLockService struct { + lockservice.LockService + closeCalls int +} + +func (s *ddlVisibilityCloseLockService) Close() error { + s.closeCalls++ + return nil +} + +type ddlVisibilityCloseTestState struct { + service *service + hakeeperClient *ddlVisibilityWithdrawalHAKeeperClient + queryService *closeRecordingQueryService + lockService *ddlVisibilityCloseLockService + cluster *ddlVisibilityTestCluster + queryClosedBeforeRefresh bool +} + +func newDDLVisibilityCloseTestService(t *testing.T, sendErr error) *ddlVisibilityCloseTestState { + t.Helper() + const generation = uint64(7) + moruntime.SetupServiceBasedRuntime(t.Name(), moruntime.DefaultRuntime()) + gossipNode, err := gossip.NewNode(context.Background(), t.Name()) + require.NoError(t, err) + cfg := &Config{UUID: t.Name()} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + state := &ddlVisibilityCloseTestState{ + queryService: &closeRecordingQueryService{closed: make(chan struct{})}, + lockService: &ddlVisibilityCloseLockService{}, + cluster: &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: t.Name(), QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + }}}, + } + state.cluster.refreshHook = func() { + select { + case <-state.queryService.closed: + state.queryClosedBeforeRefresh = true + default: + } + } + state.hakeeperClient = &ddlVisibilityWithdrawalHAKeeperClient{ + cluster: state.cluster, queryClosed: state.queryService.closed, sendErr: sendErr, + } + state.service = &service{ + cfg: cfg, + logger: zap.NewNop(), + stopper: stopper.NewStopper("ddl-visibility-close-test"), + _hakeeperClient: state.hakeeperClient, + moCluster: state.cluster, + queryService: state.queryService, + mo: closeErrorMOServer{}, + cancelMoServerFunc: func() {}, + server: closeOnlyRPCServer{}, + lockService: state.lockService, + gossipNode: gossipNode, + config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + state.service.viewMetadataIngressReady.Store(true) + state.service.ddlVisibilityBarrierReady.Store(true) + return state +} + +func TestPrepareDDLVisibilityBarrier(t *testing.T) { + const serviceID = "ddl-visibility-startup-test" + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + targetTS := timestamp.Timestamp{PhysicalTime: 200, LogicalTime: 3} + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + { + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: 7, DDLVisibilityBarrierReady: true, + }, + { + ServiceID: "peer", QueryAddress: "peer:6001", + ViewMetadataAdmissionGeneration: 9, DDLVisibilityBarrierReady: true, + }, + }, frontiers: map[string]timestamp.Timestamp{"peer": targetTS}} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{ + "self:6001": {PhysicalTime: 100}, + "peer:6001": targetTS, + }, + } + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().GetLatestCommitTS().Return(targetTS).AnyTimes() + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS).Return(targetTS.Next(), nil) + txnClient.EXPECT().SyncLatestCommitTS(targetTS) + + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + metadataFS := newDDLVisibilityMetadataFS(t) + writer := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: metadataFS, + } + // Persist with the old process, then construct a distinct service and load + // through the production metadata initialization path. + require.NoError(t, writer.persistDDLVisibilityDeployedProtocol(defines.MORPCVersion44)) + s := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: metadataFS, + logger: zap.NewNop(), + moCluster: cluster, queryClient: queryClient, _txnClient: txnClient, + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: 7, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + require.NoError(t, s.initMetadata()) + require.Equal(t, defines.MORPCVersion44, s.metadata.DDLVisibilityDeployedProtocol) + require.False(t, s.ddlVisibilityActivationComplete.Load()) + require.NoError(t, s.prepareDDLVisibilityBarrier()) + require.True(t, s.ddlVisibilityBarrierPrepared.Load()) + require.True(t, s.ddlVisibilityActivationPrepared.Load()) + require.True(t, s.ddlVisibilityActivationFenced.Load()) + require.True(t, s.ddlVisibilityActivationComplete.Load()) + require.True(t, cluster.phases[serviceID].DDLVisibilityActivationPrepared) + require.True(t, cluster.phases[serviceID].DDLVisibilityActivationFenced, + "a restarted committed CN must publish proof for its new incarnation") + require.True(t, s.ddlVisibilityBarrierReady.Load()) + require.Equal(t, 1, cluster.refreshCalls) + require.Empty(t, queryClient.requests) + require.Empty(t, queryClient.methods) + require.Zero(t, queryClient.releases) + require.NoError(t, s.publishDDLVisibilityIngressAfterStart()) + version, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.True(t, s.viewMetadataIngressReady.Load()) + require.True(t, s.ddlCommitGate.PublicDDLEnabled()) +} + +func TestDefaultV43StartupUsesLastDeployedProtocolUntilActivation(t *testing.T) { + const serviceID = "ddl-visibility-default-v43-startup-protocol-test" + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + gate := frontend.NewDDLCommitGate() + s := &service{cfg: &Config{UUID: serviceID}, ddlCommitGate: gate} + + require.NoError(t, s.prepareDDLVisibilityBarrier()) + require.NoError(t, s.publishDDLVisibilityIngressAfterStart()) + version, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion42, version) + require.True(t, s.ddlVisibilityListenersReady.Load()) + require.True(t, s.viewMetadataIngressReady.Load()) + require.True(t, gate.PublicDDLEnabled()) + require.False(t, s.ddlVisibilityActivationPrepared.Load()) + require.False(t, s.ddlVisibilityActivationFenced.Load()) +} + +func TestDDLVisibilityEpochHAKeeperCapabilityRejectsOldLeaderView(t *testing.T) { + client := &ddlVisibilityWithdrawalHAKeeperClient{} + s := &service{ + cfg: &Config{}, _hakeeperClient: client, + } + s.cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + require.NoError(t, s.requireDDLVisibilityEpochHAKeeperCapability(context.Background())) + + // A leader failover to an old RSM returns the additive field as false. The + // activation entry point must interpret that as unsupported, never epoch 0. + client.oldHAKeeperReplica = true + err := s.requireDDLVisibilityEpochHAKeeperCapability(context.Background()) + require.ErrorContains(t, err, "does not support the durable DDL visibility deployment epoch") +} + +func TestActivationFinalMembershipScanRejectsConcurrentJoin(t *testing.T) { + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + {ServiceID: "target-cn", QueryAddress: "target:6001", ViewMetadataAdmissionGeneration: 1, + DDLVisibilityBarrierReady: true}, + {ServiceID: "joining-cn", QueryAddress: "joining:6001", ViewMetadataAdmissionGeneration: 2, + DDLVisibilityBarrierReady: true}, + }} + s := &service{_hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}} + _, err := s.revalidateDDLVisibilityActivationMembership( + context.Background(), map[string]struct{}{"target-cn": {}}) + require.ErrorContains(t, err, "authoritative CN joining-cn is not fenced") +} + +func TestMarkerlessCNIngressHandshakeLinearizesWithCommittedCut(t *testing.T) { + const serviceID = "ddl-visibility-markerless-join-race-test" + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ServiceID: serviceID}}} + client := &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster} + gate := frontend.NewDDLCommitGate() + s := &service{ + cfg: &Config{UUID: serviceID}, ddlCommitGate: gate, + _hakeeperClient: client, config: util.NewConfigData(nil), + } + + // The startup read precedes the cut, but the atomic ingress heartbeat lands + // after it. HAKeeper must reject ingress and return the committed epoch. + require.NoError(t, s.prepareDDLVisibilityBarrier()) + client.clusterDeployedProtocol = defines.MORPCVersion44 + require.NoError(t, s.publishDDLVisibilityIngressAfterStart()) + version, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.False(t, s.viewMetadataIngressReady.Load()) + require.False(t, cluster.cnServices[0].ViewMetadataIngressReady) + require.False(t, gate.PublicDDLEnabled()) +} + +func TestMarkerlessCNJoinsCommittedClusterFailClosed(t *testing.T) { + const serviceID = "ddl-visibility-markerless-post-cut-test" + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + cluster := &ddlVisibilityTestCluster{} + gate := frontend.NewDDLCommitGate() + s := &service{ + cfg: &Config{UUID: serviceID}, ddlCommitGate: gate, config: util.NewConfigData(nil), + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{ + cluster: cluster, clusterDeployedProtocol: defines.MORPCVersion44, + }, + } + + require.NoError(t, s.prepareDDLVisibilityBarrier()) + require.NoError(t, s.publishDDLVisibilityIngressAfterStart()) + version, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.False(t, s.ddlVisibilityActivationComplete.Load()) + require.False(t, s.viewMetadataIngressReady.Load()) + require.False(t, gate.PublicDDLEnabled()) +} + +func TestProvisionalV43RestartRemainsFailClosed(t *testing.T) { + const serviceID = "ddl-visibility-provisional-v43-restart-test" + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + metadataFS := newDDLVisibilityMetadataFS(t) + cfg := &Config{UUID: serviceID} + writer := &service{cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: metadataFS} + require.NoError(t, writer.persistDDLVisibilityDeployedProtocol(-defines.MORPCVersion44)) + + gate := frontend.NewDDLCommitGate() + restarted := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: metadataFS, + logger: zap.NewNop(), ddlCommitGate: gate, + } + require.NoError(t, restarted.initMetadata()) + require.NoError(t, restarted.prepareDDLVisibilityBarrier()) + require.NoError(t, restarted.publishDDLVisibilityIngressAfterStart()) + version, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.False(t, restarted.ddlVisibilityActivationComplete.Load()) + require.False(t, restarted.viewMetadataIngressReady.Load()) + require.False(t, gate.PublicDDLEnabled()) +} + +func TestPrepareDDLVisibilityBarrierRejectsMissingProductionDependencies(t *testing.T) { + const serviceID = "ddl-visibility-missing-dependency-test" + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + s := &service{ + cfg: &Config{UUID: serviceID}, metadata: metadata.CNStore{UUID: serviceID}, + metadataFS: newDDLVisibilityMetadataFS(t), viewMetadataAdmissionGeneration: 1, + } + require.NoError(t, s.persistDDLVisibilityDeployedProtocol(defines.MORPCVersion44)) + + err := s.prepareDDLVisibilityBarrier() + require.ErrorContains(t, err, "dependencies are unavailable") + require.False(t, s.ddlVisibilityBarrierPrepared.Load()) + require.False(t, s.ddlVisibilityBarrierReady.Load()) +} + +func TestPrepareDDLVisibilityBarrierSkipsFrontierSyncDuringRollingUpgrade(t *testing.T) { + const serviceID = "ddl-visibility-mixed-version-test" + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + + s := &service{cfg: &Config{UUID: serviceID}} + require.NoError(t, s.prepareDDLVisibilityBarrier()) + require.True(t, s.ddlVisibilityBarrierPrepared.Load()) + require.False(t, s.ddlVisibilityActivationPrepared.Load()) + require.False(t, s.ddlVisibilityActivationFenced.Load()) + require.True(t, s.ddlVisibilityBarrierReady.Load()) +} + +func TestHandleSetProtocolVersionRejectsStaleRecoveryIdentity(t *testing.T) { + const serviceID = "ddl-visibility-stale-recovery-identity-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + s := &service{ + cfg: &Config{UUID: serviceID}, viewMetadataAdmissionGeneration: generation, + } + err := s.handleSetProtocolVersion(context.Background(), &query.Request{ + SetProtocolVersion: &query.SetProtocolVersionRequest{ + Version: defines.MORPCVersion44, DDLVisibilityActivationTargets: []string{serviceID}, + DDLVisibilityTargetGeneration: generation + 1, + DDLVisibilityTargetQueryAddress: "stale:6001", + }, + }, &query.Response{}, nil) + require.ErrorContains(t, err, "stale DDL visibility activation target identity") + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion34, version) +} + +func TestActivationDoesNotPublishFencedBeforeProvisionalPersistence(t *testing.T) { + const serviceID = "ddl-visibility-provisional-persist-failure-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion40) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + ViewMetadataIngressReady: true, + }}} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{"self:6001": {PhysicalTime: 100}}, + } + txnClient := mock_frontend.NewMockTxnClient(gomock.NewController(t)) + txnClient.EXPECT().GetLatestCommitTS().Return(timestamp.Timestamp{}) + baseFS := newDDLVisibilityMetadataFS(t) + metadataFS := &failReplaceMetadataFS{ReplaceableFileService: baseFS, failAt: 1} + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + s := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: metadataFS, + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + moCluster: cluster, queryClient: queryClient, _txnClient: txnClient, + config: util.NewConfigData(nil), viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + err := s.setProtocolVersion(context.Background(), defines.MORPCVersion44, []string{serviceID}) + require.ErrorContains(t, err, "injected metadata replace failure") + require.True(t, s.ddlVisibilityActivationPrepared.Load()) + require.False(t, s.ddlVisibilityActivationFenced.Load()) + require.False(t, s.ddlVisibilityActivationComplete.Load()) + require.False(t, s.viewMetadataIngressReady.Load()) + require.Equal(t, int64(0), s.loadDDLVisibilityDeployedProtocol()) +} + +func TestCommittedPersistenceFailureRestartsFromProvisionalFailClosed(t *testing.T) { + const serviceID = "ddl-visibility-committed-persist-failure-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion40) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + ViewMetadataIngressReady: true, + }}} + targetTS := timestamp.Timestamp{PhysicalTime: 100} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{"self:6001": targetTS}, + } + txnClient := mock_frontend.NewMockTxnClient(gomock.NewController(t)) + txnClient.EXPECT().GetLatestCommitTS().Return(targetTS) + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS).Return(targetTS.Next(), nil) + txnClient.EXPECT().SyncLatestCommitTS(targetTS) + baseFS := newDDLVisibilityMetadataFS(t) + metadataFS := &failReplaceMetadataFS{ReplaceableFileService: baseFS, failAt: 2} + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + s := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: metadataFS, + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + moCluster: cluster, queryClient: queryClient, _txnClient: txnClient, + config: util.NewConfigData(nil), viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + err := s.setProtocolVersion(context.Background(), defines.MORPCVersion44, []string{serviceID}) + require.ErrorContains(t, err, "injected metadata replace failure") + require.True(t, s.ddlVisibilityActivationFenced.Load()) + require.False(t, s.ddlVisibilityActivationComplete.Load()) + require.False(t, s.viewMetadataIngressReady.Load()) + require.Equal(t, -defines.MORPCVersion44, s.loadDDLVisibilityDeployedProtocol()) + + // A distinct process reloads the provisional marker through initMetadata and + // must remain a v43, non-routable producer until activation is retried. + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + restarted := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: baseFS, + logger: zap.NewNop(), ddlCommitGate: frontend.NewDDLCommitGate(), + } + require.NoError(t, restarted.initMetadata()) + require.NoError(t, restarted.prepareDDLVisibilityBarrier()) + require.NoError(t, restarted.publishDDLVisibilityIngressAfterStart()) + version, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.False(t, restarted.viewMetadataIngressReady.Load()) + require.False(t, restarted.ddlCommitGate.PublicDDLEnabled()) +} + +func TestActivationWithdrawalFailureStillBlocksNewDDL(t *testing.T) { + const serviceID = "ddl-visibility-withdrawal-failure-gate-test" + const generation = uint64(7) + withdrawErr := errors.New("injected withdrawal failure") + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion35) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + ViewMetadataIngressReady: true, + }}} + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + gate := frontend.NewDDLCommitGate() + s := &service{ + cfg: cfg, _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{ + cluster: cluster, sendErr: withdrawErr, + }, + moCluster: cluster, queryClient: &ddlVisibilityTestQueryClient{serviceID: serviceID}, + _txnClient: mock_frontend.NewMockTxnClient(gomock.NewController(t)), + config: util.NewConfigData(nil), viewMetadataAdmissionGeneration: generation, + ddlCommitGate: gate, + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + err := s.setProtocolVersion(context.Background(), defines.MORPCVersion44, []string{serviceID}) + require.ErrorIs(t, err, withdrawErr) + require.True(t, cluster.cnServices[0].ViewMetadataIngressReady, + "the injected heartbeat failure leaves authoritative ingress unchanged") + blockedCtx, cancel := context.WithCancel(context.Background()) + cancel() + _, err = gate.Enter(blockedCtx) + require.ErrorIs(t, err, context.Canceled, + "withdrawal failure must not reopen old-protocol DDL admission") +} + +func TestActivationDrainTimeoutKeepsIngressWithdrawn(t *testing.T) { + const serviceID = "ddl-visibility-drain-timeout-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion35) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + ViewMetadataIngressReady: true, + }}} + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = 10 * time.Millisecond + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + gate := frontend.NewDDLCommitGate() + releaseActive, err := gate.Enter(context.Background()) + require.NoError(t, err) + defer releaseActive() + s := &service{ + cfg: cfg, _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + moCluster: cluster, queryClient: &ddlVisibilityTestQueryClient{serviceID: serviceID}, + _txnClient: mock_frontend.NewMockTxnClient(gomock.NewController(t)), + config: util.NewConfigData(nil), viewMetadataAdmissionGeneration: generation, + ddlCommitGate: gate, + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + err = s.setProtocolVersion(context.Background(), defines.MORPCVersion44, []string{serviceID}) + require.Error(t, err) + require.True(t, s.ddlVisibilityActivationPending.Load()) + require.False(t, s.viewMetadataIngressReady.Load()) + require.False(t, cluster.cnServices[0].ViewMetadataIngressReady) + blockedCtx, cancel := context.WithCancel(context.Background()) + cancel() + _, err = gate.Enter(blockedCtx) + require.Error(t, err) +} + +func TestDefaultV43StillRunsCompleteTargetActivation(t *testing.T) { + const serviceID = "ddl-visibility-default-v43-activation-test" + const generation = uint64(7) + moruntime.SetupServiceBasedRuntime(serviceID, moruntime.DefaultRuntime()) + targetTS := timestamp.Timestamp{PhysicalTime: 300, LogicalTime: 4} + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + {ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + ViewMetadataIngressReady: true}, + {ServiceID: "legacy-peer", QueryAddress: "peer:6001", + ViewMetadataAdmissionGeneration: 8, DDLVisibilityBarrierReady: false, + ViewMetadataIngressReady: true}, + }, frontiers: map[string]timestamp.Timestamp{"legacy-peer": targetTS}} + cluster.refreshHook = func() { + // The control plane dispatches activation concurrently. Model the legacy + // peer completing its local drain before this CN checks global phases. + cluster.cnServices[1].DDLVisibilityBarrierReady = true + } + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{ + "self:6001": {PhysicalTime: 100}, "peer:6001": targetTS, + }, + protocols: map[string]query.GetProtocolVersionResponse{"peer:6001": { + Version: defines.MORPCVersion44, + DDLVisibilityActivationPrepared: true, DDLVisibilityActivationFenced: true, + }}, + } + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().GetLatestCommitTS().Return(targetTS).AnyTimes() + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS).Return(targetTS.Next(), nil) + txnClient.EXPECT().SyncLatestCommitTS(targetTS) + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + s := &service{ + cfg: cfg, metadata: metadata.CNStore{UUID: serviceID}, metadataFS: newDDLVisibilityMetadataFS(t), + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + moCluster: cluster, queryClient: queryClient, _txnClient: txnClient, + config: util.NewConfigData(nil), viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.ddlVisibilityActivationPrepared.Store(true) + s.ddlVisibilityActivationFenced.Store(true) + s.viewMetadataIngressReady.Store(true) + + require.False(t, s.ddlVisibilityActivationComplete.Load()) + require.NoError(t, s.setProtocolVersion( + context.Background(), defines.MORPCVersion44, []string{serviceID, "legacy-peer"})) + require.True(t, s.ddlVisibilityActivationComplete.Load()) + require.Equal(t, defines.MORPCVersion44, s.loadDDLVisibilityDeployedProtocol()) + require.True(t, s.viewMetadataIngressReady.Load()) + require.True(t, s.ddlCommitGate.PublicDDLEnabled()) + require.Empty(t, queryClient.requests) +} + +func TestHandleSetProtocolVersionFencesRunningCNActivation(t *testing.T) { + const serviceID = "ddl-visibility-live-activation-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + targetTS := timestamp.Timestamp{PhysicalTime: 300, LogicalTime: 4} + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + { + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + }, + { + ServiceID: "peer", QueryAddress: "peer:6001", + ViewMetadataAdmissionGeneration: 9, DDLVisibilityBarrierReady: true, + }, + }} + cluster.frontiers = map[string]timestamp.Timestamp{"peer": targetTS} + var peerReady atomic.Bool + var protocolObserved atomic.Bool + firstProtocol := make(chan struct{}) + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{ + "self:6001": {PhysicalTime: 200}, + "peer:6001": targetTS, + }, + protocolFn: func(string) query.GetProtocolVersionResponse { + if protocolObserved.CompareAndSwap(false, true) { + close(firstProtocol) + } + ready := peerReady.Load() + return query.GetProtocolVersionResponse{ + Version: defines.MORPCVersion44, + DDLVisibilityActivationPrepared: ready, + DDLVisibilityActivationFenced: ready, + } + }, + } + cluster.phaseHook = queryClient.protocolFn + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().GetLatestCommitTS().Return(targetTS).AnyTimes() + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS).DoAndReturn( + func(context.Context, timestamp.Timestamp) (timestamp.Timestamp, error) { + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version, + "sender capability must be v43 after every local v34 DDL producer is drained") + require.True(t, cluster.cnServices[0].DDLVisibilityBarrierReady, + "the transitioning CN must remain reachable by v43 DDL fan-out") + require.False(t, cluster.cnServices[0].ViewMetadataIngressReady, + "the transitioning CN must not accept new proxy sessions before the fence") + return targetTS.Next(), nil + }) + txnClient.EXPECT().SyncLatestCommitTS(targetTS) + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + hakeeperClient := &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster} + ddlCommitGate := frontend.NewDDLCommitGate() + s := &service{ + cfg: cfg, + _hakeeperClient: hakeeperClient, + moCluster: cluster, + queryClient: queryClient, + _txnClient: txnClient, + config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: generation, + ddlCommitGate: ddlCommitGate, + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + req := &query.Request{SetProtocolVersion: &query.SetProtocolVersionRequest{ + Version: defines.MORPCVersion44, + DDLVisibilityActivationTargets: []string{serviceID, "peer"}, + DDLVisibilityTargetGeneration: generation, + DDLVisibilityTargetQueryAddress: "self:6001", + }} + resp := &query.Response{} + activationDone := make(chan error, 1) + go func() { + activationDone <- s.handleSetProtocolVersion(context.Background(), req, resp, nil) + }() + <-firstProtocol + blockedCtx, cancelBlocked := context.WithCancel(context.Background()) + cancelBlocked() + _, err := ddlCommitGate.Enter(blockedCtx) + require.ErrorIs(t, err, context.Canceled, + "DDL producer must remain blocked until every target is prepared and fenced") + peerReady.Store(true) + require.NoError(t, <-activationDone) + releaseDDL, err := ddlCommitGate.Enter(context.Background()) + require.NoError(t, err, "activation must release DDL producers after global convergence") + releaseDDL() + require.Equal(t, defines.MORPCVersion44, resp.SetProtocolVersion.Version) + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.Empty(t, queryClient.requests) + require.Empty(t, queryClient.methods) + require.Zero(t, queryClient.releases) + require.Len(t, hakeeperClient.heartbeats, 5) + require.False(t, hakeeperClient.heartbeats[0].ViewMetadataIngressReady) + require.True(t, hakeeperClient.heartbeats[1].DDLVisibilityActivationPrepared) + require.False(t, hakeeperClient.heartbeats[1].DDLVisibilityActivationFenced) + require.True(t, hakeeperClient.heartbeats[2].DDLVisibilityActivationFenced) + require.NotEmpty(t, hakeeperClient.heartbeats[3].DDLVisibilityEpochCommitTargets) + require.False(t, hakeeperClient.heartbeats[3].ViewMetadataIngressReady) + require.True(t, hakeeperClient.heartbeats[4].ViewMetadataIngressReady) + require.Equal(t, 2, cluster.refreshCalls) + require.True(t, s.ddlVisibilityBarrierReady.Load()) + require.True(t, s.viewMetadataIngressReady.Load()) + + // Repeating an already-active version must not withdraw or re-run the fence. + resp = &query.Response{} + require.NoError(t, s.handleSetProtocolVersion(context.Background(), req, resp, nil)) + require.Equal(t, defines.MORPCVersion44, resp.SetProtocolVersion.Version) + require.Len(t, hakeeperClient.heartbeats, 5) + require.Equal(t, 2, cluster.refreshCalls) + require.Empty(t, queryClient.requests) +} + +func TestHandleSetProtocolVersionPreservesPreStartIngress(t *testing.T) { + const serviceID = "ddl-visibility-pre-start-ingress-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + {ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true}, + {ServiceID: "peer", QueryAddress: "peer:6001", + ViewMetadataAdmissionGeneration: 8, DDLVisibilityBarrierReady: true}, + }} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{"self:6001": {}, "peer:6001": {}}, + protocols: activationTestPeerProtocols(), + } + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + hakeeperClient := &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster} + txnClient := mock_frontend.NewMockTxnClient(gomock.NewController(t)) + txnClient.EXPECT().GetLatestCommitTS().Return(timestamp.Timestamp{}).AnyTimes() + s := &service{ + cfg: cfg, _hakeeperClient: hakeeperClient, moCluster: cluster, + queryClient: queryClient, _txnClient: txnClient, + config: util.NewConfigData(nil), viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(false) + + resp := &query.Response{} + require.NoError(t, s.handleSetProtocolVersion(context.Background(), &query.Request{ + SetProtocolVersion: &query.SetProtocolVersionRequest{ + Version: defines.MORPCVersion44, + DDLVisibilityActivationTargets: []string{serviceID, "peer"}, + DDLVisibilityTargetGeneration: generation, DDLVisibilityTargetQueryAddress: "self:6001", + }, + }, resp, nil)) + require.Equal(t, defines.MORPCVersion44, resp.SetProtocolVersion.Version) + require.False(t, s.viewMetadataIngressReady.Load()) + require.False(t, cluster.cnServices[0].ViewMetadataIngressReady) + require.Len(t, hakeeperClient.heartbeats, 5) + require.False(t, hakeeperClient.heartbeats[0].ViewMetadataIngressReady) + require.True(t, hakeeperClient.heartbeats[1].DDLVisibilityActivationPrepared) + require.True(t, hakeeperClient.heartbeats[2].DDLVisibilityActivationFenced) + require.NotEmpty(t, hakeeperClient.heartbeats[3].DDLVisibilityEpochCommitTargets) + require.False(t, hakeeperClient.heartbeats[4].ViewMetadataIngressReady) + release, err := s.ddlCommitGate.Enter(context.Background()) + require.NoError(t, err) + release() + + // Re-run the transition and linearize listener-ready publication while + // activation still owns the barrier lock. The later Start publication must + // win; activation must not restore its stale pre-listener false sample. + serviceRuntime := moruntime.ServiceRuntime(serviceID) + serviceRuntime.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + value, ok := serviceRuntime.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion34, value) + s.ddlVisibilityActivationPending.Store(true) + s.ddlVisibilityActivationPrepared.Store(false) + s.ddlVisibilityActivationFenced.Store(false) + s.viewMetadataIngressReady.Store(false) + cluster.cnServices[0].ViewMetadataIngressReady = false + phaseEntered := make(chan struct{}) + allowPhase := make(chan struct{}) + var phaseCalls atomic.Int32 + queryClient.protocolFn = func(string) query.GetProtocolVersionResponse { + if phaseCalls.Add(1) == 1 { + close(phaseEntered) + <-allowPhase + } + return query.GetProtocolVersionResponse{ + Version: defines.MORPCVersion44, + DDLVisibilityActivationPrepared: true, DDLVisibilityActivationFenced: true, + } + } + cluster.phaseHook = queryClient.protocolFn + activationDone := make(chan error, 1) + go func() { + activationDone <- s.handleSetProtocolVersion(context.Background(), &query.Request{ + SetProtocolVersion: &query.SetProtocolVersionRequest{ + Version: defines.MORPCVersion44, + DDLVisibilityActivationTargets: []string{serviceID, "peer"}, + DDLVisibilityTargetGeneration: generation, DDLVisibilityTargetQueryAddress: "self:6001", + }, + }, &query.Response{}, nil) + }() + select { + case <-phaseEntered: + case err := <-activationDone: + require.NoError(t, err) + t.Fatal("activation completed before entering phase convergence") + } + publishDone := make(chan error, 1) + go func() { publishDone <- s.publishDDLVisibilityIngressAfterStart() }() + close(allowPhase) + require.NoError(t, <-activationDone) + require.NoError(t, <-publishDone) + require.True(t, s.viewMetadataIngressReady.Load()) +} + +func TestHandleSetProtocolVersionFailsClosedWhenActivationSyncFails(t *testing.T) { + const serviceID = "ddl-visibility-live-activation-failure-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + targetTS := timestamp.Timestamp{PhysicalTime: 300, LogicalTime: 4} + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + { + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + }, + {ServiceID: "peer", QueryAddress: "peer:6001", ViewMetadataAdmissionGeneration: 8, + DDLVisibilityBarrierReady: true}, + }} + cluster.frontiers = map[string]timestamp.Timestamp{"peer": targetTS} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{ + "self:6001": {PhysicalTime: 200}, "peer:6001": targetTS, + }, + protocols: activationTestPeerProtocols(), + } + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().GetLatestCommitTS().Return(targetTS) + syncErr := errors.New("logtail activation sync failed") + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS).Return(timestamp.Timestamp{}, syncErr) + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + hakeeperClient := &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster} + s := &service{ + cfg: cfg, + _hakeeperClient: hakeeperClient, + moCluster: cluster, + queryClient: queryClient, + _txnClient: txnClient, + config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + resp := &query.Response{} + err := s.handleSetProtocolVersion(context.Background(), &query.Request{ + SetProtocolVersion: &query.SetProtocolVersionRequest{ + Version: defines.MORPCVersion44, DDLVisibilityActivationTargets: []string{serviceID, "peer"}, + DDLVisibilityTargetGeneration: generation, DDLVisibilityTargetQueryAddress: "self:6001", + }, + }, resp, nil) + require.ErrorIs(t, err, syncErr) + require.Nil(t, resp.SetProtocolVersion) + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.True(t, s.ddlVisibilityActivationPending.Load()) + require.True(t, s.ddlVisibilityActivationPrepared.Load()) + require.False(t, s.ddlVisibilityActivationFenced.Load()) + blockedCtx, cancelBlocked := context.WithCancel(context.Background()) + cancelBlocked() + _, gateErr := s.ddlCommitGate.Enter(blockedCtx) + require.ErrorIs(t, gateErr, context.Canceled) + require.Len(t, hakeeperClient.heartbeats, 2) + require.True(t, hakeeperClient.heartbeats[0].DDLVisibilityBarrierReady) + require.True(t, hakeeperClient.heartbeats[1].DDLVisibilityActivationPrepared) + require.False(t, s.viewMetadataIngressReady.Load()) + require.True(t, cluster.cnServices[0].DDLVisibilityBarrierReady) +} + +func TestHandleSetProtocolVersionWithdrawsAfterActivationPublishFails(t *testing.T) { + const serviceID = "ddl-visibility-live-activation-publish-failure-test" + const generation = uint64(7) + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + targetTS := timestamp.Timestamp{PhysicalTime: 300, LogicalTime: 4} + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + { + ServiceID: serviceID, QueryAddress: "self:6001", + ViewMetadataAdmissionGeneration: generation, DDLVisibilityBarrierReady: true, + }, + {ServiceID: "peer", QueryAddress: "peer:6001", ViewMetadataAdmissionGeneration: 8, + DDLVisibilityBarrierReady: true}, + }} + cluster.frontiers = map[string]timestamp.Timestamp{"peer": targetTS} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{ + "self:6001": {PhysicalTime: 200}, "peer:6001": targetTS, + }, + protocols: activationTestPeerProtocols(), + } + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().GetLatestCommitTS().Return(targetTS).AnyTimes() + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS).Return(targetTS.Next(), nil) + txnClient.EXPECT().SyncLatestCommitTS(targetTS) + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.DiscoveryTimeout.Duration = time.Second + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + publishErr := errors.New("activation publication failed") + hakeeperClient := &ddlVisibilityWithdrawalHAKeeperClient{ + cluster: cluster, sendErrors: map[int]error{5: publishErr}, + } + s := &service{ + cfg: cfg, + _hakeeperClient: hakeeperClient, + moCluster: cluster, + queryClient: queryClient, + _txnClient: txnClient, + config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: generation, + ddlCommitGate: frontend.NewDDLCommitGate(), + } + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + err := s.handleSetProtocolVersion(context.Background(), &query.Request{ + SetProtocolVersion: &query.SetProtocolVersionRequest{ + Version: defines.MORPCVersion44, DDLVisibilityActivationTargets: []string{serviceID, "peer"}, + DDLVisibilityTargetGeneration: generation, DDLVisibilityTargetQueryAddress: "self:6001", + }, + }, &query.Response{}, nil) + require.ErrorIs(t, err, publishErr) + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.True(t, s.ddlVisibilityActivationPending.Load()) + require.True(t, s.ddlVisibilityActivationFenced.Load()) + blockedCtx, cancelBlocked := context.WithCancel(context.Background()) + cancelBlocked() + _, gateErr := s.ddlCommitGate.Enter(blockedCtx) + require.ErrorIs(t, gateErr, context.Canceled) + require.Len(t, hakeeperClient.heartbeats, 6) + require.True(t, hakeeperClient.heartbeats[0].DDLVisibilityBarrierReady) + require.True(t, hakeeperClient.heartbeats[1].DDLVisibilityActivationPrepared) + require.True(t, hakeeperClient.heartbeats[2].DDLVisibilityActivationFenced) + require.NotEmpty(t, hakeeperClient.heartbeats[3].DDLVisibilityEpochCommitTargets) + require.True(t, hakeeperClient.heartbeats[4].ViewMetadataIngressReady) + require.False(t, hakeeperClient.heartbeats[5].ViewMetadataIngressReady) + require.False(t, s.viewMetadataIngressReady.Load()) + require.True(t, cluster.cnServices[0].DDLVisibilityBarrierReady) + require.Equal(t, 2, cluster.refreshCalls) +} + +func TestSetProtocolVersionBeforeBarrierPreparationDefersToStartupFence(t *testing.T) { + const serviceID = "ddl-visibility-pre-start-activation-test" + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + s := &service{cfg: &Config{UUID: serviceID}} + + require.NoError(t, s.setProtocolVersion(context.Background(), defines.MORPCVersion44, nil)) + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + require.False(t, s.ddlVisibilityBarrierPrepared.Load()) + require.False(t, s.ddlVisibilityBarrierReady.Load()) +} + +func TestInitQueryCommandHandlerOverridesProtocolActivation(t *testing.T) { + const serviceID = "ddl-visibility-public-protocol-handler-test" + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + queryService := &closeRecordingQueryService{ + handlers: make(map[query.CmdMethod]func(context.Context, *query.Request, *query.Response, *morpc.Buffer) error), + } + s := &service{cfg: &Config{UUID: serviceID}, queryService: queryService} + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierClosing.Store(true) + s.ddlVisibilityActivationPrepared.Store(true) + s.ddlVisibilityActivationFenced.Store(true) + s.initQueryCommandHandler() + + getHandler, ok := queryService.handlers[query.CmdMethod_GetProtocolVersion] + require.True(t, ok) + getResp := &query.Response{} + require.NoError(t, getHandler(context.Background(), &query.Request{ + GetProtocolVersion: &query.GetProtocolVersionRequest{}, + }, getResp, nil)) + require.Equal(t, defines.MORPCVersion34, getResp.GetProtocolVersion.Version) + require.True(t, getResp.GetProtocolVersion.DDLVisibilityActivationPrepared) + require.True(t, getResp.GetProtocolVersion.DDLVisibilityActivationFenced) + require.ErrorContains(t, + getHandler(context.Background(), &query.Request{}, &query.Response{}, nil), + "bad request") + rt.SetGlobalVariables(moruntime.MOProtocolVersion, "invalid") + require.ErrorContains(t, getHandler(context.Background(), &query.Request{ + GetProtocolVersion: &query.GetProtocolVersionRequest{}, + }, &query.Response{}, nil), "invalid protocol version") + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + + handler, ok := queryService.handlers[query.CmdMethod_SetProtocolVersion] + require.True(t, ok) + resp := &query.Response{} + err := handler(context.Background(), &query.Request{ + SetProtocolVersion: &query.SetProtocolVersionRequest{Version: defines.MORPCVersion44}, + }, resp, nil) + require.ErrorContains(t, err, "CN is closing") + require.Nil(t, resp.SetProtocolVersion) + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion34, version) +} + +func TestSetProtocolVersionRejectsActivationDuringClose(t *testing.T) { + const serviceID = "ddl-visibility-closing-activation-test" + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion34) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + s := &service{cfg: &Config{UUID: serviceID}} + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityBarrierClosing.Store(true) + + err := s.setProtocolVersion(context.Background(), defines.MORPCVersion44, nil) + require.ErrorContains(t, err, "CN is closing") + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion34, version) +} + +func TestWithdrawDDLVisibilityBarrierHandlesPartialService(t *testing.T) { + t.Run("zero generation has no authoritative publication", func(t *testing.T) { + s := &service{} + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + require.NoError(t, s.withdrawDDLVisibilityBarrierLocked(context.Background())) + require.False(t, s.ddlVisibilityBarrierReady.Load()) + require.False(t, s.viewMetadataIngressReady.Load()) + }) + + t.Run("published generation requires dependencies", func(t *testing.T) { + s := &service{cfg: &Config{}, viewMetadataAdmissionGeneration: 1} + err := s.withdrawDDLVisibilityBarrierLocked(context.Background()) + require.ErrorContains(t, err, "withdrawal dependencies are unavailable") + }) +} + +func TestValidateDDLVisibilityActivationTargets(t *testing.T) { + _, err := validateDDLVisibilityActivationTargets("self", []string{"self", ""}) + require.ErrorContains(t, err, "empty") + _, err = validateDDLVisibilityActivationTargets("self", []string{"self", "self"}) + require.ErrorContains(t, err, "duplicated") + _, err = validateDDLVisibilityActivationTargets("self", []string{"peer"}) + require.ErrorContains(t, err, "do not include local CN") + targets, err := validateDDLVisibilityActivationTargets("self", []string{"self", "peer"}) + require.NoError(t, err) + require.Equal(t, map[string]struct{}{"self": {}, "peer": {}}, targets) +} + +func TestSetProtocolVersionDowngradeGuards(t *testing.T) { + const serviceID = "ddl-visibility-downgrade-test" + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion44) + moruntime.SetupServiceBasedRuntime(serviceID, rt) + s := &service{cfg: &Config{UUID: serviceID}} + s.ddlVisibilityBarrierPrepared.Store(true) + s.ddlVisibilityActivationPrepared.Store(true) + s.ddlVisibilityActivationFenced.Store(true) + s.ddlVisibilityActivationComplete.Store(true) + + err := s.setProtocolVersion(context.Background(), defines.MORPCVersion40, nil) + require.ErrorContains(t, err, "cannot downgrade after the DDL visibility cluster epoch is committed") + version, ok := rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion44, version) + + s.ddlVisibilityActivationComplete.Store(false) + s._hakeeperClient = &ddlVisibilityWithdrawalHAKeeperClient{ + clusterDeployedProtocol: defines.MORPCVersion44, + } + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion40) + err = s.setProtocolVersion(context.Background(), defines.MORPCVersion34, nil) + require.ErrorContains(t, err, "cannot downgrade after the DDL visibility cluster epoch is committed") + + s._hakeeperClient = nil + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion40) + require.NoError(t, s.setProtocolVersion(context.Background(), defines.MORPCVersion34, nil)) + version, ok = rt.GetGlobalVariables(moruntime.MOProtocolVersion) + require.True(t, ok) + require.Equal(t, defines.MORPCVersion34, version) + + s.ddlVisibilityActivationPending.Store(true) + err = s.setProtocolVersion(context.Background(), defines.MORPCVersion34, nil) + require.ErrorContains(t, err, "cannot downgrade during DDL visibility activation") +} + +func TestWaitForDDLVisibilityActivationPhaseRejectsInvalidInventory(t *testing.T) { + const serviceID = "activation-phase-inventory-test" + newService := func(cluster *ddlVisibilityTestCluster) *service { + cfg := &Config{UUID: serviceID} + cfg.HAKeeper.HeatbeatInterval.Duration = time.Millisecond + s := &service{cfg: cfg, moCluster: cluster, + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}} + s.ddlVisibilityActivationPrepared.Store(true) + s.ddlVisibilityActivationFenced.Store(true) + return s + } + targets := map[string]struct{}{serviceID: {}, "peer": {}} + + t.Run("target address is required", func(t *testing.T) { + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + {ServiceID: serviceID, DDLVisibilityBarrierReady: true}, + {ServiceID: "peer", DDLVisibilityBarrierReady: true}, + }} + s := newService(cluster) + err := s.waitForDDLVisibilityActivationPhase(context.Background(), targets, false) + require.ErrorContains(t, err, "has no authoritative identity") + }) + + t.Run("target list must include every barrier participant", func(t *testing.T) { + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{ + {ServiceID: serviceID, QueryAddress: "self:6001", ViewMetadataAdmissionGeneration: 7, + DDLVisibilityBarrierReady: true}, + {ServiceID: "peer", QueryAddress: "peer:6001", ViewMetadataAdmissionGeneration: 8, + DDLVisibilityBarrierReady: true}, + {ServiceID: "extra", QueryAddress: "extra:6001", ViewMetadataAdmissionGeneration: 9, + DDLVisibilityBarrierReady: true}, + }} + s := newService(cluster) + err := s.waitForDDLVisibilityActivationPhase(context.Background(), targets, false) + require.ErrorContains(t, err, "omits authoritative CN") + }) +} + +func TestWaitForDDLVisibilityIngressHonorsCancellation(t *testing.T) { + const serviceID = "ddl-visibility-ingress-cancel-test" + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, ViewMetadataAdmissionGeneration: 1, + DDLVisibilityBarrierReady: true, ViewMetadataIngressReady: true, + }}} + s := &service{ + cfg: &Config{UUID: serviceID}, moCluster: cluster, + viewMetadataAdmissionGeneration: 1, + } + ctx, cancel := context.WithTimeout(context.Background(), time.Millisecond) + defer cancel() + err := s.waitForDDLVisibilityIngress(ctx, time.Second, false) + require.ErrorContains(t, err, "was not published") +} + +func TestWaitForDDLVisibilityBarrierPublicationHonorsCancellation(t *testing.T) { + cluster := &ddlVisibilityTestCluster{} + s := &service{ + cfg: &Config{UUID: "unpublished-cn"}, + moCluster: cluster, + viewMetadataAdmissionGeneration: 1, + } + ctx, cancel := context.WithTimeout(context.Background(), time.Millisecond) + defer cancel() + + err := s.waitForDDLVisibilityBarrierPublication(ctx, time.Second) + require.Error(t, err) + require.Equal(t, 1, cluster.refreshCalls) +} + +func TestPeriodicHeartbeatCannotRepublishStaleIngressDuringActivation(t *testing.T) { + client := &ddlVisibilityHeartbeatOrderClient{ + firstStarted: make(chan struct{}), releaseFirst: make(chan struct{}), + } + cfg := &Config{UUID: "ddl-visibility-heartbeat-order-test"} + cfg.HAKeeper.HeatbeatTimeout.Duration = time.Second + s := &service{ + cfg: cfg, logger: zap.NewNop(), _hakeeperClient: client, + config: util.NewConfigData(nil), + } + s.ddlVisibilityBarrierReady.Store(true) + s.viewMetadataIngressReady.Store(true) + + heartbeatDone := make(chan struct{}) + go func() { + s.heartbeat(context.Background()) + close(heartbeatDone) + }() + <-client.firstStarted + activationStarted := make(chan struct{}) + activationDone := make(chan struct{}) + go func() { + close(activationStarted) + s.ddlVisibilityHeartbeatMu.Lock() + s.viewMetadataIngressReady.Store(false) + _, _ = client.SendCNHeartbeat(context.Background(), s.newCNStoreHeartbeat()) + s.ddlVisibilityHeartbeatMu.Unlock() + close(activationDone) + }() + <-activationStarted + close(client.releaseFirst) + <-heartbeatDone + <-activationDone + + client.mu.Lock() + defer client.mu.Unlock() + require.Len(t, client.completed, 2) + require.True(t, client.completed[0].ViewMetadataIngressReady) + require.False(t, client.completed[1].ViewMetadataIngressReady, + "the activation withdrawal must complete after every older true heartbeat") +} + +func TestServiceCloseWithdrawsDDLVisibilityBeforeQueryService(t *testing.T) { + state := newDDLVisibilityCloseTestService(t, nil) + + require.NoError(t, state.service.Close()) + require.True(t, state.hakeeperClient.queryClosedBeforeSend, + "view-admission withdrawal intentionally follows QueryService drain") + require.False(t, state.queryClosedBeforeRefresh) + require.Len(t, state.hakeeperClient.heartbeats, 2) + require.False(t, state.hakeeperClient.heartbeats[0].DDLVisibilityBarrierReady) + require.False(t, state.hakeeperClient.heartbeats[0].ViewMetadataIngressReady, + "shutdown seals local view ingress before either authoritative withdrawal") + require.False(t, state.hakeeperClient.heartbeats[1].DDLVisibilityBarrierReady) + require.False(t, state.hakeeperClient.heartbeats[1].ViewMetadataIngressReady) + require.Equal(t, 1, state.cluster.refreshCalls) + require.False(t, state.cluster.cnServices[0].DDLVisibilityBarrierReady) + require.Equal(t, 1, state.hakeeperClient.closeCalls) + require.Equal(t, 2, state.lockService.closeCalls) + _, err := state.service.ddlCommitGate.Enter(context.Background()) + require.ErrorContains(t, err, "closed") + select { + case <-state.queryService.closed: + default: + t.Fatal("QueryService was not closed after authoritative barrier withdrawal") + } +} + +func TestServiceCloseContinuesAfterDDLVisibilityWithdrawalFailure(t *testing.T) { + withdrawErr := errors.New("withdraw heartbeat failed") + state := newDDLVisibilityCloseTestService(t, withdrawErr) + + err := state.service.Close() + require.ErrorIs(t, err, withdrawErr) + require.True(t, state.hakeeperClient.queryClosedBeforeSend, + "clean view-admission withdrawal is still attempted after the DDL withdrawal failure") + require.Len(t, state.hakeeperClient.heartbeats, 2) + require.Zero(t, state.cluster.refreshCalls) + require.Equal(t, 1, state.hakeeperClient.closeCalls) + require.Equal(t, 2, state.lockService.closeCalls) + select { + case <-state.queryService.closed: + default: + t.Fatal("QueryService cleanup was skipped after barrier withdrawal failure") + } +} + +func TestWaitForDDLVisibilityBarrierWithdrawalAcceptsNewerGeneration(t *testing.T) { + const serviceID = "ddl-visibility-newer-generation-test" + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, ViewMetadataAdmissionGeneration: 8, + DDLVisibilityBarrierReady: true, + }}} + s := &service{ + cfg: &Config{UUID: serviceID}, + moCluster: cluster, + viewMetadataAdmissionGeneration: 7, + } + + require.NoError(t, s.waitForDDLVisibilityBarrierWithdrawal(context.Background(), time.Second)) + require.Equal(t, 1, cluster.refreshCalls) +} + +func TestWaitForDDLVisibilityBarrierWithdrawalHonorsCancellation(t *testing.T) { + const serviceID = "ddl-visibility-withdrawal-timeout-test" + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, ViewMetadataAdmissionGeneration: 7, + DDLVisibilityBarrierReady: true, + }}} + s := &service{ + cfg: &Config{UUID: serviceID}, + moCluster: cluster, + viewMetadataAdmissionGeneration: 7, + } + ctx, cancel := context.WithTimeout(context.Background(), time.Millisecond) + defer cancel() + + err := s.waitForDDLVisibilityBarrierWithdrawal(ctx, time.Second) + require.Error(t, err) + require.Equal(t, 1, cluster.refreshCalls) +} + +func TestAcknowledgedDDLFrontierSurvivesCrashBeforePeriodicHeartbeat(t *testing.T) { + const producerID = "ddl-producer" + frontier := timestamp.Timestamp{PhysicalTime: 300} + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: producerID, QueryAddress: "old:6001", + ViewMetadataAdmissionGeneration: 1, DDLVisibilityBarrierReady: true, + }}} + client := &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster} + producer := &service{ + cfg: &Config{UUID: producerID}, _hakeeperClient: client, + ddlCommitGate: frontend.NewDDLCommitGate(), config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: 1, + } + + // This is the synchronous publication in the successful DDL commit path; + // deliberately do not run a periodic heartbeat before replacing the CN. + producer.ddlCommitGate.RecordDDLFrontier(frontier) + require.NoError(t, producer.publishDDLCommitFrontier(context.Background(), frontier)) + require.Equal(t, frontier, cluster.globalFrontier) + + // A fresh same-UUID incarnation starts with an empty process-local gate. + replacement := &service{ + cfg: &Config{UUID: producerID}, _hakeeperClient: client, + ddlCommitGate: frontend.NewDDLCommitGate(), config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: 2, + } + require.NoError(t, replacement.publishDDLCommitFrontier(context.Background(), timestamp.Timestamp{})) + require.Equal(t, frontier, cluster.globalFrontier, + "replacement must not erase the synchronously published commit frontier") + + ctrl := gomock.NewController(t) + laggingTxnClient := mock_frontend.NewMockTxnClient(ctrl) + laggingTxnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), frontier).Return(frontier, nil) + laggingTxnClient.EXPECT().SyncLatestCommitTS(frontier) + target := &service{ + cfg: &Config{UUID: "lagging-peer"}, _txnClient: laggingTxnClient, + _hakeeperClient: client, + } + require.NoError(t, target.syncStartupDDLVisibilityFrontier(context.Background())) +} + +func TestLegacyHAKeeperCannotAcknowledgeDroppedDDLFrontier(t *testing.T) { + const producerID = "legacy-hakeeper-ddl-producer" + oldFrontier := timestamp.Timestamp{PhysicalTime: 200} + commitFrontier := timestamp.Timestamp{PhysicalTime: 300} + cluster := &ddlVisibilityTestCluster{ + cnServices: []metadata.CNService{{ + ServiceID: producerID, QueryAddress: "producer:6001", + ViewMetadataAdmissionGeneration: 1, DDLVisibilityBarrierReady: true, + }}, + globalFrontier: oldFrontier, + } + client := &ddlVisibilityWithdrawalHAKeeperClient{ + cluster: cluster, oldDDLFrontierRSM: true, + } + producer := &service{ + cfg: &Config{UUID: producerID}, _hakeeperClient: client, + ddlCommitGate: frontend.NewDDLCommitGate(), config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: 1, logger: zap.NewNop(), + } + producer.ddlCommitGate.SetFrontierPublisher(producer.publishDDLCommitFrontier) + + err := producer.ddlCommitGate.PublishDDLFrontier(context.Background(), commitFrontier) + require.ErrorContains(t, err, "was not durably acknowledged") + require.Equal(t, oldFrontier, cluster.globalFrontier) + + ctrl := gomock.NewController(t) + restartedTxnClient := mock_frontend.NewMockTxnClient(ctrl) + restartedTxnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), oldFrontier).Return(oldFrontier, nil) + restartedTxnClient.EXPECT().SyncLatestCommitTS(oldFrontier) + restarted := &service{ + cfg: &Config{UUID: producerID}, _txnClient: restartedTxnClient, + _hakeeperClient: client, + } + require.NoError(t, restarted.syncStartupDDLVisibilityFrontier(context.Background())) +} + +func TestStaleGenerationCannotAcknowledgeDDLFrontierPublication(t *testing.T) { + const producerID = "generation-fenced-ddl-producer" + oldFrontier := timestamp.Timestamp{PhysicalTime: 200} + commitFrontier := timestamp.Timestamp{PhysicalTime: 300} + cluster := &ddlVisibilityTestCluster{ + cnServices: []metadata.CNService{{ + ServiceID: producerID, QueryAddress: "new:6001", + ViewMetadataAdmissionGeneration: 2, DDLVisibilityBarrierReady: true, + }}, + globalFrontier: oldFrontier, + } + client := &ddlVisibilityWithdrawalHAKeeperClient{ + cluster: cluster, + authoritativeGenerations: map[string]uint64{producerID: 2}, + } + oldIncarnation := &service{ + cfg: &Config{UUID: producerID}, _hakeeperClient: client, + ddlCommitGate: frontend.NewDDLCommitGate(), config: util.NewConfigData(nil), + viewMetadataAdmissionGeneration: 1, logger: zap.NewNop(), + } + runner := &blockedStopTaskRunner{ + testRunner: &testRunner{}, stopEntered: make(chan struct{}), + releaseStop: make(chan struct{}), stopDone: make(chan struct{}), + } + oldIncarnation.task.runner = runner + oldIncarnation.task.runnerReady.Store(true) + oldIncarnation.ddlCommitGate.SetFrontierPublisher(oldIncarnation.publishDDLCommitFrontier) + + // Model the stale publication running inside the TaskRunner SQL task. The + // rejection must return before asynchronous runner.Stop can observe this task + // exit; otherwise each side waits permanently for the other. + publishDone := make(chan error, 1) + go func() { + publishDone <- oldIncarnation.ddlCommitGate.PublishDDLFrontier( + context.Background(), commitFrontier) + }() + var err error + select { + case err = <-publishDone: + case <-time.After(time.Second): + t.Fatal("generation rejection self-waited on its own TaskRunner SQL task") + } + require.ErrorContains(t, err, "generation 1 rejected by authoritative generation 2") + select { + case <-runner.stopEntered: + case <-time.After(time.Second): + t.Fatal("asynchronous revocation did not begin TaskRunner drain") + } + close(runner.releaseStop) + select { + case <-runner.stopDone: + case <-time.After(time.Second): + t.Fatal("TaskRunner drain did not finish after the rejected SQL task exited") + } + require.True(t, oldIncarnation.viewMetadataGenerationRevoked.Load()) + require.Equal(t, oldFrontier, cluster.globalFrontier, + "stale generation heartbeat must not advance the durable frontier") + + // After the stale process crashes, a future incarnation can only observe the + // old durable value. The failed publication therefore must not have been + // acknowledged by frontend (covered by the publisher-failure commit test). + ctrl := gomock.NewController(t) + restartedTxnClient := mock_frontend.NewMockTxnClient(ctrl) + restartedTxnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), oldFrontier).Return(oldFrontier, nil) + restartedTxnClient.EXPECT().SyncLatestCommitTS(oldFrontier) + restarted := &service{ + cfg: &Config{UUID: producerID}, _txnClient: restartedTxnClient, + _hakeeperClient: client, + } + require.NoError(t, restarted.syncStartupDDLVisibilityFrontier(context.Background())) +} + +func TestSyncStartupDDLVisibilityFrontierSurvivesProducerReplacement(t *testing.T) { + const serviceID = "ddl-visibility-durable-frontier-test" + oldFrontier := timestamp.Timestamp{PhysicalTime: 300} + cluster := &ddlVisibilityTestCluster{ + cnServices: []metadata.CNService{{ + ServiceID: "replaced-producer", QueryAddress: "new:6001", + ViewMetadataAdmissionGeneration: 2, DDLVisibilityBarrierReady: true, + }}, + globalFrontier: oldFrontier, + } + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), oldFrontier).Return(oldFrontier, nil) + txnClient.EXPECT().SyncLatestCommitTS(oldFrontier) + s := &service{ + cfg: &Config{UUID: serviceID}, _txnClient: txnClient, + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + } + + require.NoError(t, s.syncStartupDDLVisibilityFrontier(context.Background())) +} + +func TestSyncStartupDDLVisibilityFrontierAllowsEmptyFrontier(t *testing.T) { + const serviceID = "ddl-visibility-empty-frontier-test" + cluster := &ddlVisibilityTestCluster{cnServices: []metadata.CNService{{ + ServiceID: serviceID, QueryAddress: "self:6001", + DDLVisibilityBarrierReady: true, + }}} + queryClient := &ddlVisibilityTestQueryClient{ + serviceID: serviceID, + frontiers: map[string]timestamp.Timestamp{"self:6001": {}}, + } + s := &service{ + cfg: &Config{UUID: serviceID}, moCluster: cluster, queryClient: queryClient, + _hakeeperClient: &ddlVisibilityWithdrawalHAKeeperClient{cluster: cluster}, + } + + require.NoError(t, s.syncStartupDDLVisibilityFrontier(context.Background())) + require.Empty(t, queryClient.requests) + require.Zero(t, queryClient.releases) +} diff --git a/pkg/cnservice/server_heartbeat.go b/pkg/cnservice/server_heartbeat.go index 4bda50a6edfd6..1bff76e96345e 100644 --- a/pkg/cnservice/server_heartbeat.go +++ b/pkg/cnservice/server_heartbeat.go @@ -22,9 +22,11 @@ import ( "github.com/matrixorigin/matrixone/pkg/common/moerr" "github.com/matrixorigin/matrixone/pkg/common/system" + "github.com/matrixorigin/matrixone/pkg/defines" "github.com/matrixorigin/matrixone/pkg/logservice" "github.com/matrixorigin/matrixone/pkg/logutil" logservicepb "github.com/matrixorigin/matrixone/pkg/pb/logservice" + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" v2 "github.com/matrixorigin/matrixone/pkg/util/metric/v2" "github.com/matrixorigin/matrixone/pkg/version" ) @@ -185,6 +187,59 @@ func (s *service) notifyCommandPoll() { } } +// sendCNHeartbeat transfers schedule-command ownership to the existing dedupe +// owner before returning. Direct publication heartbeats must not discard a +// destructively delivered legacy CommandBatch. +func (s *service) sendCNHeartbeat( + ctx context.Context, + hb logservicepb.CNStoreHeartbeat, +) (logservicepb.CommandBatch, error) { + batch, err := s._hakeeperClient.SendCNHeartbeat(ctx, hb) + if err != nil { + return batch, err + } + s.handleHeartbeatResponse(hb.AckedCommandBatchID, batch) + return batch, nil +} + +// publishDDLCommitFrontier durably advances HAKeeper's cluster frontier before +// frontend acknowledges a public pre-cut DDL. It is serialized with periodic +// heartbeat snapshots so an older captured heartbeat cannot follow this one. +func (s *service) publishDDLCommitFrontier(ctx context.Context, ts timestamp.Timestamp) error { + if s._hakeeperClient == nil { + return moerr.NewInternalError(ctx, "HAKeeper client is unavailable while publishing DDL frontier") + } + s.ddlVisibilityHeartbeatMu.Lock() + defer s.ddlVisibilityHeartbeatMu.Unlock() + batch, err := s.sendCNHeartbeat(ctx, s.newCNStoreHeartbeat()) + if err != nil { + return moerr.AttachCause(ctx, err) + } + admission := batch.ViewMetadataAdmission + if s.viewMetadataAdmissionGeneration == 0 || admission == nil { + return moerr.NewInvalidStateNoCtx( + "DDL frontier publication lacks an authoritative CN generation") + } + if admission.Generation != s.viewMetadataAdmissionGeneration { + if admission.Generation > s.viewMetadataAdmissionGeneration { + s.revokeViewMetadataGeneration(admission.Generation) + } + return moerr.NewInvalidStateNoCtxf( + "DDL frontier publication generation %d rejected by authoritative generation %d", + s.viewMetadataAdmissionGeneration, admission.Generation) + } + if batch.DDLVisibilityFrontier == nil || batch.DDLVisibilityFrontier.Less(ts) { + authoritative := "missing" + if batch.DDLVisibilityFrontier != nil { + authoritative = batch.DDLVisibilityFrontier.DebugString() + } + return moerr.NewInvalidStateNoCtxf( + "DDL frontier publication %s was not durably acknowledged (authoritative frontier %s)", + ts.DebugString(), authoritative) + } + return nil +} + func (s *service) newCNStoreHeartbeat() logservicepb.CNStoreHeartbeat { hb := logservicepb.CNStoreHeartbeat{ UUID: s.cfg.UUID, @@ -210,6 +265,15 @@ func (s *service) newCNStoreHeartbeat() logservicepb.CNStoreHeartbeat { ViewMetadataAdmissionGeneration: s.viewMetadataAdmissionGeneration, ViewMetadataCatalogFencedEpoch: s.viewMetadataCatalogFencedEpoch.Load(), ViewMetadataIngressReady: s.viewMetadataIngressReady.Load(), + DDLVisibilityBarrierReady: s.ddlVisibilityBarrierReady.Load(), + DDLVisibilityActivationPrepared: s.ddlVisibilityActivationPrepared.Load(), + DDLVisibilityActivationFenced: s.ddlVisibilityActivationFenced.Load(), + } + if s.ddlCommitGate != nil { + hb.DDLVisibilityFrontier = s.ddlCommitGate.LatestDDLFrontier() + } + if deployed := s.loadDDLVisibilityDeployedProtocol(); deployed >= defines.MORPCVersion44 { + hb.DDLVisibilityDeployedProtocol = deployed } if s.viewMetadataEpochFence != nil { hb.ViewMetadataObservedEpoch = s.viewMetadataEpochFence.Epoch() @@ -222,10 +286,8 @@ func (s *service) newCNStoreHeartbeat() logservicepb.CNStoreHeartbeat { } // withdrawViewMetadataAdmission publishes a final non-routable heartbeat after -// the periodic heartbeat task and every ingress path have stopped. This is the -// clean ownership-handoff point: a replacement can take the same UUID without -// waiting for store timeout, while abrupt or partitioned processes still have -// no acknowledgement and remain protected by the timeout fence. +// every local ingress path and TaskRunner have stopped. A clean replacement can +// then take the UUID without waiting for the store timeout. func (s *service) withdrawViewMetadataAdmission() error { if s.viewMetadataAdmissionGeneration == 0 || s._hakeeperClient == nil { return nil @@ -239,7 +301,7 @@ func (s *service) withdrawViewMetadataAdmission() error { context.Background(), timeout, moerr.CauseHeartbeat) defer cancel() hb := s.newCNStoreHeartbeat() - _, err := s._hakeeperClient.SendCNHeartbeat(ctx, hb) + _, err := s.sendCNHeartbeat(ctx, hb) if err != nil { return moerr.AttachCause(ctx, err) } @@ -258,12 +320,16 @@ func (s *service) heartbeat(ctx context.Context) { ctx2, cancel := context.WithTimeoutCause(ctx, s.cfg.HAKeeper.HeatbeatTimeout.Duration, moerr.CauseHeartbeat) defer cancel() + // Snapshot and send readiness under the same owner as startup, activation, + // and shutdown publication. Otherwise an older in-flight true heartbeat can + // arrive after activation has authoritatively published ingress=false. + s.ddlVisibilityHeartbeatMu.Lock() hb := s.newCNStoreHeartbeat() - s.heartbeatInFlight.Store(true) s.notifyCommandPoll() cb, err := s._hakeeperClient.SendCNHeartbeat(ctx2, hb) s.heartbeatInFlight.Store(false) + s.ddlVisibilityHeartbeatMu.Unlock() if err != nil { s.commandPollNeeded.Store(true) s.notifyCommandPoll() diff --git a/pkg/cnservice/server_heartbeat_test.go b/pkg/cnservice/server_heartbeat_test.go index 733d3f5e54acb..639aaa2563df9 100644 --- a/pkg/cnservice/server_heartbeat_test.go +++ b/pkg/cnservice/server_heartbeat_test.go @@ -207,6 +207,26 @@ func (c *blockingCNHeartbeatCommandClient) GetScheduleCommands( return c.commandBatch, nil } +func TestDirectCNHeartbeatProcessesLegacyScheduleCommand(t *testing.T) { + holder := &observingTaskHolder{createErr: errors.New("stop after observing command"), created: make(chan struct{}, 1)} + client := &admissionFailureCNHeartbeatClient{ + testHAKClient: &testHAKClient{}, + batch: pb.CommandBatch{Commands: []pb.ScheduleCommand{{ + ServiceType: pb.CNService, + CreateTaskService: &pb.CreateTaskService{}, + }}}, + } + s := &service{ + cfg: &Config{UUID: "direct-heartbeat-command-owner"}, config: util.NewConfigData(nil), + logger: zap.NewNop(), _hakeeperClient: client, + } + s.task.holder = holder + + _, err := s.sendCNHeartbeat(context.Background(), s.newCNStoreHeartbeat()) + require.NoError(t, err) + require.Equal(t, int32(1), holder.createCount.Load()) +} + func Test_heartbeat(t *testing.T) { ctx, cancel := context.WithCancel(context.Background()) defer cancel() diff --git a/pkg/cnservice/server_query.go b/pkg/cnservice/server_query.go index 59f492cd238b8..98e0359d31c2c 100644 --- a/pkg/cnservice/server_query.go +++ b/pkg/cnservice/server_query.go @@ -140,6 +140,11 @@ func (s *service) initQueryService() error { } func (s *service) initQueryCommandHandler() { + // Override queryservice's generic runtime-only setter so a live CN can + // execute the v36 DDL visibility activation fence before acknowledging the + // protocol transition. + s.addQueryCommandHandler(query.CmdMethod_GetProtocolVersion, s.handleGetProtocolVersion) + s.addQueryCommandHandler(query.CmdMethod_SetProtocolVersion, s.handleSetProtocolVersion) s.addQueryCommandHandler(query.CmdMethod_KillConn, s.handleKillConn) s.addQueryCommandHandler(query.CmdMethod_AlterAccount, s.handleAlterAccount) s.addQueryCommandHandler(query.CmdMethod_TraceSpan, s.handleTraceSpan) @@ -147,6 +152,7 @@ func (s *service) initQueryCommandHandler() { s.addQueryCommandHandler(query.CmdMethod_GetTxnInfo, s.handleGetTxnInfo) s.addQueryCommandHandler(query.CmdMethod_GetCacheInfo, s.handleGetCacheInfo) s.addQueryCommandHandler(query.CmdMethod_SyncCommit, s.handleSyncCommit) + s.addQueryCommandHandler(query.CmdMethod_SyncCommitV2, s.handleSyncCommit) s.addQueryCommandHandler(query.CmdMethod_GetCommit, s.handleGetCommit) s.addQueryCommandHandler(query.CmdMethod_ShowProcessList, s.handleShowProcessList) s.addQueryCommandHandler(query.CmdMethod_RunTask, s.handleRunTask) @@ -204,6 +210,59 @@ func (s *service) closeQueryService() error { }) } +func (s *service) handleGetProtocolVersion( + ctx context.Context, + req *query.Request, + resp *query.Response, + _ *morpc.Buffer, +) error { + if req == nil || req.GetProtocolVersion == nil { + return moerr.NewInternalError(ctx, "bad request") + } + value, ok := moruntime.ServiceRuntime(s.cfg.UUID).GetGlobalVariables(moruntime.MOProtocolVersion) + if !ok { + return moerr.NewInternalError(ctx, "protocol version not found") + } + version, ok := value.(int64) + if !ok { + return moerr.NewInternalError(ctx, "invalid protocol version") + } + resp.GetProtocolVersion = &query.GetProtocolVersionResponse{ + Version: version, + DDLVisibilityActivationPrepared: s.ddlVisibilityActivationPrepared.Load(), + DDLVisibilityActivationFenced: s.ddlVisibilityActivationFenced.Load(), + } + return nil +} + +func (s *service) handleSetProtocolVersion( + ctx context.Context, + req *query.Request, + resp *query.Response, + _ *morpc.Buffer, +) error { + if req == nil || req.SetProtocolVersion == nil { + return moerr.NewInternalError(ctx, "bad request") + } + version := req.SetProtocolVersion.Version + if version >= defines.MORPCVersion44 && + len(req.SetProtocolVersion.DDLVisibilityActivationTargets) > 0 { + expectedGeneration := req.SetProtocolVersion.DDLVisibilityTargetGeneration + expectedAddress := req.SetProtocolVersion.DDLVisibilityTargetQueryAddress + localAddress := s.queryServiceServiceAddr() + if expectedGeneration == 0 || expectedGeneration != s.viewMetadataAdmissionGeneration || + expectedAddress == "" || (localAddress != "" && expectedAddress != localAddress) { + return moerr.NewInvalidStateNoCtx("stale DDL visibility activation target identity") + } + } + if err := s.setProtocolVersion( + ctx, version, req.SetProtocolVersion.DDLVisibilityActivationTargets); err != nil { + return err + } + resp.SetProtocolVersion = &query.SetProtocolVersionResponse{Version: version} + return nil +} + func (s *service) handleKillConn(ctx context.Context, req *query.Request, resp *query.Response, _ *morpc.Buffer) error { if req == nil || req.KillConnRequest == nil { return moerr.NewInternalError(ctx, "bad request") @@ -445,8 +504,23 @@ func (s *service) handleGetTxnInfo(ctx context.Context, req *query.Request, resp return nil } +const DDLVisibilitySyncCommitFaultPoint = "ddl_visibility_sync_commit_error" + func (s *service) handleSyncCommit(ctx context.Context, req *query.Request, resp *query.Response, _ *morpc.Buffer) error { - s._txnClient.SyncLatestCommitTS(req.SycnCommit.LatestCommitTS) + if _, msg, injected := fault.TriggerFault(DDLVisibilitySyncCommitFaultPoint); injected { + if msg == "" { + msg = DDLVisibilitySyncCommitFaultPoint + } + return moerr.NewInternalErrorNoCtxf("injected DDL visibility sync commit error: %s", msg) + } + targetTS := req.SycnCommit.LatestCommitTS + if _, err := s._txnClient.WaitLogTailAppliedAt(ctx, targetTS); err != nil { + return err + } + // Preserve the existing CN-based latest-commit update and BVT counter. The + // timestamp wait above makes this call non-blocking while keeping it on the + // established SyncCommit path. + s._txnClient.SyncLatestCommitTS(targetTS) return nil } diff --git a/pkg/cnservice/server_query_test.go b/pkg/cnservice/server_query_test.go index bb881375b135e..9541978cb57d9 100644 --- a/pkg/cnservice/server_query_test.go +++ b/pkg/cnservice/server_query_test.go @@ -53,6 +53,7 @@ import ( "github.com/matrixorigin/matrixone/pkg/pb/query" "github.com/matrixorigin/matrixone/pkg/pb/statsinfo" "github.com/matrixorigin/matrixone/pkg/pb/task" + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" "github.com/matrixorigin/matrixone/pkg/queryservice" "github.com/matrixorigin/matrixone/pkg/shardservice" sqlmongodb "github.com/matrixorigin/matrixone/pkg/sql/mongodb" @@ -67,6 +68,45 @@ import ( var dummyBadRequestErr = moerr.NewInternalError(context.TODO(), "bad request") var dummyErr = moerr.NewInternalError(context.TODO(), "dummy error") +func TestServiceHandleSyncCommitWaitsForAppliedLogtail(t *testing.T) { + targetTS := timestamp.Timestamp{PhysicalTime: 100, LogicalTime: 7} + + t.Run("updates latest commit after logtail is applied", func(t *testing.T) { + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + txnClient.EXPECT().WaitLogTailAppliedAt(gomock.Any(), targetTS). + Return(targetTS.Next(), nil) + txnClient.EXPECT().SyncLatestCommitTS(targetTS) + + s := &service{_txnClient: txnClient} + err := s.handleSyncCommit( + context.Background(), + &query.Request{SycnCommit: &query.SyncCommitRequest{LatestCommitTS: targetTS}}, + &query.Response{}, + nil, + ) + require.NoError(t, err) + }) + + t.Run("propagates cancellation without updating latest commit", func(t *testing.T) { + ctrl := gomock.NewController(t) + txnClient := mock_frontend.NewMockTxnClient(ctrl) + ctx, cancel := context.WithCancel(context.Background()) + cancel() + txnClient.EXPECT().WaitLogTailAppliedAt(ctx, targetTS). + Return(timestamp.Timestamp{}, context.Canceled) + + s := &service{_txnClient: txnClient} + err := s.handleSyncCommit( + ctx, + &query.Request{SycnCommit: &query.SyncCommitRequest{LatestCommitTS: targetTS}}, + &query.Response{}, + nil, + ) + require.ErrorIs(t, err, context.Canceled) + }) +} + func Test_service_handleISCPDrainConsumerRenewFenceOnly(t *testing.T) { exec := &iscp.ISCPTaskExecutor{} iscp.RegisterExecutorRuntime("runner-cn", exec) diff --git a/pkg/cnservice/server_task.go b/pkg/cnservice/server_task.go index cb4a071d76272..322d6940143a5 100644 --- a/pkg/cnservice/server_task.go +++ b/pkg/cnservice/server_task.go @@ -213,22 +213,32 @@ func (s *service) publishTaskRunner() error { return nil } -func (s *service) detachRevokedTaskRunner() taskservice.TaskRunner { +func (s *service) detachRevokedTaskRunner() (taskservice.TaskRunner, chan struct{}) { s.task.Lock() defer s.task.Unlock() s.task.generationRevoked = true s.task.runnerReady.Store(false) - runner := s.task.runner - s.task.runner = nil - return runner + if s.task.runner != nil { + s.task.revokedRunner = s.task.runner + s.task.revokedRunnerDone = make(chan struct{}) + s.task.runner = nil + } + return s.task.revokedRunner, s.task.revokedRunnerDone } -func (s *service) stopRevokedTaskRunner(runner taskservice.TaskRunner) { - if runner != nil { - if err := runner.Stop(); err != nil { - s.logger.Error("stop revoked generation task runner failed", zap.Error(err)) - } +func (s *service) stopRevokedTaskRunner(runner taskservice.TaskRunner, done chan struct{}) { + if runner == nil { + return + } + if err := runner.Stop(); err != nil && s.logger != nil { + s.logger.Error("stop revoked generation task runner failed", zap.Error(err)) + } + s.task.Lock() + if s.task.revokedRunner == runner { + s.task.revokedRunner = nil + close(done) } + s.task.Unlock() } func (s *service) startTaskRunnerLocked() { @@ -304,15 +314,25 @@ func (s *service) GetTaskService() (taskservice.TaskService, bool) { func (s *service) stopTask() error { defer logutil.LogClose(s.logger, "cnservice/task")() + // Transfer normal runner ownership while holding task.Lock, then release it + // before any Stop/wait. A SQL task may synchronously trigger generation + // revocation and must be able to acquire this lock in order to exit. s.task.Lock() - defer s.task.Unlock() - if s.task.holder == nil { - return nil - } + holder := s.task.holder + runner := s.task.runner + s.task.runner = nil + revokedDone := s.task.revokedRunnerDone + s.task.Unlock() - err := s.task.holder.Close() - if s.task.runner != nil { - err = errors.Join(err, s.task.runner.Stop()) + var err error + if holder != nil { + err = holder.Close() + } + if runner != nil { + err = errors.Join(err, runner.Stop()) + } + if revokedDone != nil { + <-revokedDone } return err } diff --git a/pkg/cnservice/server_view_metadata_admission.go b/pkg/cnservice/server_view_metadata_admission.go index fa7332e792af8..830aaf37489bc 100644 --- a/pkg/cnservice/server_view_metadata_admission.go +++ b/pkg/cnservice/server_view_metadata_admission.go @@ -24,6 +24,7 @@ import ( "github.com/matrixorigin/matrixone/pkg/common/runtime" logservicepb "github.com/matrixorigin/matrixone/pkg/pb/logservice" "github.com/matrixorigin/matrixone/pkg/sql/compile" + "github.com/matrixorigin/matrixone/pkg/taskservice" ) const viewMetadataAdmissionGenerationKey = "view-metadata-admission-generation" @@ -114,45 +115,55 @@ func (s *service) applyViewMetadataAdmission( } // revokeViewMetadataGeneration fences a process that no longer owns its UUID. -// The gates are closed synchronously so scheduling an asynchronous full Close -// cannot leave a window for new SQL, QueryService, or pipeline work. Full Close -// runs outside the heartbeat stopper task to avoid waiting for itself. +// The admission seal is synchronous, but every potentially waiting drain runs +// asynchronously. In particular, a SQL task can discover its own stale +// generation while publishing a DDL frontier; waiting for TaskRunner.Stop on +// that stack would make the runner wait for the task that is doing the stop. func (s *service) revokeViewMetadataGeneration(authoritative uint64) { s.viewMetadataRevocationOnce.Do(func() { s.viewMetadataGenerationRevoked.Store(true) s.viewMetadataIngressReady.Store(false) + s.ddlVisibilityBarrierReady.Store(false) _ = s.closePipelineAdmission() s.queryWork.beginClose() - runner := s.detachRevokedTaskRunner() - // Serialize physical frontend shutdown with MOServer.Start before waiting - // for task executors, whose Stop may block indefinitely. Do not wait for - // the broader lifecycleMu held by the complete Start sequence. - if s.mo != nil { - if err := s.stopFrontendSerialized(); err != nil && s.logger != nil { - s.logger.Error("failed to stop superseded CN frontend", - zap.Uint64("local-generation", s.viewMetadataAdmissionGeneration), - zap.Uint64("authoritative-generation", authoritative), - zap.Error(err)) - } - } - s.stopRevokedTaskRunner(runner) - if s.stopper != nil || s.viewMetadataCloseFn != nil { - closeFn := s.Close - if s.viewMetadataCloseFn != nil { - closeFn = s.viewMetadataCloseFn - } - go func() { - if err := closeFn(); err != nil && s.logger != nil { - s.logger.Error("failed to close superseded CN", - zap.Uint64("local-generation", s.viewMetadataAdmissionGeneration), - zap.Uint64("authoritative-generation", authoritative), - zap.Error(err)) - } - }() - } + runner, runnerDone := s.detachRevokedTaskRunner() + + go s.drainRevokedViewMetadataGeneration(authoritative, runner, runnerDone) }) } +func (s *service) drainRevokedViewMetadataGeneration( + authoritative uint64, + runner taskservice.TaskRunner, + runnerDone chan struct{}, +) { + // Serialize physical frontend shutdown with MOServer.Start before waiting + // for task executors. The goroutine is independent of the rejected SQL task, + // allowing that task to return and satisfy TaskRunner.Stop's wait. + if s.mo != nil { + if err := s.stopFrontendSerialized(); err != nil && s.logger != nil { + s.logger.Error("failed to stop superseded CN frontend", + zap.Uint64("local-generation", s.viewMetadataAdmissionGeneration), + zap.Uint64("authoritative-generation", authoritative), + zap.Error(err)) + } + } + s.stopRevokedTaskRunner(runner, runnerDone) + if s.stopper == nil && s.viewMetadataCloseFn == nil { + return + } + closeFn := s.Close + if s.viewMetadataCloseFn != nil { + closeFn = s.viewMetadataCloseFn + } + if err := closeFn(); err != nil && s.logger != nil { + s.logger.Error("failed to close superseded CN", + zap.Uint64("local-generation", s.viewMetadataAdmissionGeneration), + zap.Uint64("authoritative-generation", authoritative), + zap.Error(err)) + } +} + func (s *service) fenceViewMetadataCatalog( ctx context.Context, snapshot *logservicepb.ViewMetadataAdmission, diff --git a/pkg/cnservice/server_view_metadata_admission_test.go b/pkg/cnservice/server_view_metadata_admission_test.go index 588d576cbff19..da780ca2a8885 100644 --- a/pkg/cnservice/server_view_metadata_admission_test.go +++ b/pkg/cnservice/server_view_metadata_admission_test.go @@ -66,12 +66,15 @@ type blockedStopTaskRunner struct { *testRunner stopEntered chan struct{} releaseStop chan struct{} + stopDone chan struct{} } func (r *blockedStopTaskRunner) Stop() error { close(r.stopEntered) <-r.releaseStop - return r.testRunner.Stop() + err := r.testRunner.Stop() + close(r.stopDone) + return err } type admissionCNHAKeeperClient struct { @@ -203,9 +206,7 @@ func TestCNViewMetadataAdmissionRevokesIngressForHigherGeneration(t *testing.T) require.False(t, s.viewMetadataIngressReady.Load()) require.False(t, s.task.runnerReady.Load(), "generation revocation must synchronously withdraw task-runner eligibility") - require.Equal(t, 1, runner.stopped, - "generation revocation must synchronously stop the running task runner") - require.Nil(t, s.GetTaskRunner(), "the stopped task runner must be detached") + require.Nil(t, s.GetTaskRunner(), "the revoked task runner must be synchronously detached") select { case <-pipelineCtx.Done(): default: @@ -216,6 +217,8 @@ func TestCNViewMetadataAdmissionRevokesIngressForHigherGeneration(t *testing.T) case <-time.After(time.Second): t.Fatal("superseded CN did not request full service closure") } + require.Equal(t, 1, runner.stopped, + "asynchronous generation drain must stop the detached task runner") select { case <-mo.stopped: default: @@ -236,6 +239,7 @@ func TestCNGenerationRevocationStopsFrontendBeforeTaskRunnerDrain(t *testing.T) testRunner: &testRunner{}, stopEntered: make(chan struct{}), releaseStop: make(chan struct{}), + stopDone: make(chan struct{}), } s := &service{ cfg: &Config{UUID: "task-runner-blocked-stop"}, @@ -264,19 +268,72 @@ func TestCNGenerationRevocationStopsFrontendBeforeTaskRunnerDrain(t *testing.T) } select { case <-revokeDone: - t.Fatal("generation revocation returned before the task runner stopped") - default: + // The synchronous seal must not inherit TaskRunner.Stop's wait graph. + case <-time.After(time.Second): + t.Fatal("generation revocation seal waited for the task runner") + } + closeDone := make(chan error, 1) + go func() { closeDone <- s.stopTask() }() + select { + case <-closeDone: + t.Fatal("normal Close passed the detached runner before its drain completed") + case <-time.After(20 * time.Millisecond): } close(runner.releaseStop) select { - case <-revokeDone: + case <-runner.stopDone: case <-time.After(time.Second): - t.Fatal("generation revocation did not finish after the task runner stopped") + t.Fatal("asynchronous task runner drain did not finish") } require.Equal(t, 1, runner.stopped) + require.NoError(t, <-closeDone) require.Nil(t, s.GetTaskRunner()) } +func TestCNGenerationRevocationDoesNotWaitForCloseOwnedTaskRunner(t *testing.T) { + runner := &blockedStopTaskRunner{ + testRunner: &testRunner{}, stopEntered: make(chan struct{}), + releaseStop: make(chan struct{}), stopDone: make(chan struct{}), + } + s := &service{ + cfg: &Config{UUID: "close-owned-task-runner"}, logger: zap.NewNop(), + viewMetadataAdmissionGeneration: 8, + } + s.task.runner = runner + s.task.runnerReady.Store(true) + + closeDone := make(chan error, 1) + go func() { closeDone <- s.stopTask() }() + select { + case <-runner.stopEntered: + case <-time.After(time.Second): + t.Fatal("normal Close did not take runner drain ownership") + } + + revokeDone := make(chan struct{}) + go func() { + s.revokeViewMetadataGeneration(9) + close(revokeDone) + }() + select { + case <-revokeDone: + // stopTask must not hold task.Lock across runner.Stop. + case <-time.After(time.Second): + t.Fatal("revocation waited on task.Lock held across normal runner.Stop") + } + require.True(t, s.viewMetadataGenerationRevoked.Load()) + require.False(t, s.task.runnerReady.Load()) + + close(runner.releaseStop) + select { + case <-runner.stopDone: + case <-time.After(time.Second): + t.Fatal("normal Close runner drain did not finish") + } + require.NoError(t, <-closeDone) + require.Equal(t, 1, runner.stopped) +} + func TestCNGenerationRevocationStopsTaskRunnerStartInFlight(t *testing.T) { s := &service{ cfg: &Config{UUID: "task-runner-revoked-during-start"}, @@ -303,9 +360,8 @@ func TestCNGenerationRevocationStopsTaskRunnerStartInFlight(t *testing.T) { select { case <-revokeDone: case <-time.After(time.Second): - t.Fatal("generation revocation did not finish stopping the in-flight task runner") + t.Fatal("generation revocation did not finish sealing the in-flight task runner") } - require.Equal(t, 1, runner.stopped) require.False(t, s.task.runnerReady.Load()) require.Nil(t, s.GetTaskRunner()) } @@ -352,7 +408,7 @@ func TestCNGenerationRevocationCancelsIngressStart(t *testing.T) { require.False(t, s.task.runnerReady.Load()) select { case <-mo.stopped: - default: + case <-time.After(time.Second): t.Fatal("revocation did not stop frontend after the in-flight Start completed") } select { diff --git a/pkg/cnservice/types.go b/pkg/cnservice/types.go index 7520a3c572633..0d813af0ccb1f 100644 --- a/pkg/cnservice/types.go +++ b/pkg/cnservice/types.go @@ -799,6 +799,18 @@ type service struct { viewMetadataCatalogFenceMu sync.Mutex viewMetadataCatalogFenceReady atomic.Bool viewMetadataIngressReady atomic.Bool + ddlVisibilityBarrierReady atomic.Bool + ddlVisibilityBarrierPrepared atomic.Bool + ddlVisibilityBarrierClosing atomic.Bool + ddlVisibilityActivationPending atomic.Bool + ddlVisibilityActivationPrepared atomic.Bool + ddlVisibilityActivationFenced atomic.Bool + ddlVisibilityActivationComplete atomic.Bool + ddlVisibilityListenersReady atomic.Bool + ddlVisibilityRestoreIngress atomic.Bool + ddlVisibilityBarrierMu sync.Mutex + ddlVisibilityHeartbeatMu sync.Mutex + ddlCommitGate *frontend.DDLCommitGate viewMetadataGenerationRevoked atomic.Bool viewMetadataRevocationOnce sync.Once // viewMetadataCloseFn is a deterministic test hook for the asynchronous @@ -815,6 +827,8 @@ type service struct { sync.RWMutex holder taskservice.TaskServiceHolder runner taskservice.TaskRunner + revokedRunner taskservice.TaskRunner + revokedRunnerDone chan struct{} runnerReady atomic.Bool generationRevoked bool storageFactory taskservice.TaskStorageFactory diff --git a/pkg/defines/const.go b/pkg/defines/const.go index 50aa3a2019f44..1e84f1caacbf2 100644 --- a/pkg/defines/const.go +++ b/pkg/defines/const.go @@ -78,7 +78,9 @@ const ( MORPCVersion40 int64 = 40 // PAD SPACE comparison casts and set-operation equality keys MORPCVersion41 int64 = 41 // cycle-safe bounded current-role closure table function MORPCVersion42 int64 = 42 // transactional SQL-task child cleanup - MORPCLatestVersion = MORPCVersion42 + MORPCVersion43 int64 = 43 // validated MongoDB explicit-query payload (reserved by #27553) + MORPCVersion44 int64 = 44 // cancellable cross-CN DDL visibility fence + MORPCLatestVersion = MORPCVersion44 ) // DefaultLockWaitTimeoutSeconds is shared by the frontend default and by diff --git a/pkg/frontend/cdc_dao_sync_test.go b/pkg/frontend/cdc_dao_sync_test.go index 98d940f5dfca0..07491ac83a3bc 100644 --- a/pkg/frontend/cdc_dao_sync_test.go +++ b/pkg/frontend/cdc_dao_sync_test.go @@ -80,7 +80,11 @@ func (m *mockQueryClient) Close() error { // Mock MOCluster for testing type mockMOCluster struct { - cnServices []metadata.CNService + cnServices []metadata.CNService + refreshSnapshots [][]metadata.CNService + refreshCalls int + refreshError error + refreshErrors map[int]error } func (m *mockMOCluster) GetCNService(selector clusterservice.Selector, apply func(metadata.CNService) bool) { @@ -100,13 +104,28 @@ func (m *mockMOCluster) GetAllTNServices() []metadata.TNService { } func (m *mockMOCluster) GetCNServiceWithoutWorkingState(selector clusterservice.Selector, apply func(metadata.CNService) bool) { - // Not used in syncCommitTimestamp + for _, cn := range m.cnServices { + if !apply(cn) { + break + } + } } func (m *mockMOCluster) ForceRefresh(sync bool) { // Not used in syncCommitTimestamp } +func (m *mockMOCluster) Refresh(context.Context) error { + m.refreshCalls++ + if m.refreshCalls <= len(m.refreshSnapshots) { + m.cnServices = m.refreshSnapshots[m.refreshCalls-1] + } + if err := m.refreshErrors[m.refreshCalls]; err != nil { + return err + } + return m.refreshError +} + func (m *mockMOCluster) Close() { // Not used in syncCommitTimestamp } diff --git a/pkg/frontend/ddl_commit_gate.go b/pkg/frontend/ddl_commit_gate.go new file mode 100644 index 0000000000000..4bf59abaf055b --- /dev/null +++ b/pkg/frontend/ddl_commit_gate.go @@ -0,0 +1,245 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package frontend + +import ( + "context" + "sync" + "sync/atomic" + + "github.com/matrixorigin/matrixone/pkg/common/moerr" + moruntime "github.com/matrixorigin/matrixone/pkg/common/runtime" + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" +) + +const DDLCommitGateRuntimeKey = "frontend.ddl-commit-gate" + +// DDLCommitGate gives DDL producers and live protocol activation one local +// linearization point. Enter admits a commit while Block prevents new commits +// and waits for already-admitted commits to leave. A failed activation may keep +// the gate blocked across RPC attempts; Close wakes blocked sessions during CN +// shutdown. +type DDLCommitGate struct { + mu sync.Mutex + changed chan struct{} + blocked bool + closed bool + publicDDL bool + active int + ddlFrontier atomic.Pointer[timestamp.Timestamp] + frontierPublisher func(context.Context, timestamp.Timestamp) error + // enterBlockedHook is a deterministic test hook invoked after Enter observes + // a blocked gate and before it waits. Production leaves it nil. + enterBlockedHook func() +} + +func NewDDLCommitGate() *DDLCommitGate { + return &DDLCommitGate{changed: make(chan struct{})} +} + +func (g *DDLCommitGate) RecordDDLFrontier(ts timestamp.Timestamp) { + if ts.IsEmpty() { + return + } + candidate := ts + for { + current := g.ddlFrontier.Load() + if current != nil && current.GreaterEq(ts) { + return + } + if g.ddlFrontier.CompareAndSwap(current, &candidate) { + return + } + } +} + +// SetFrontierPublisher installs the CN-owned durable publisher before public +// sessions start. Production sets it exactly once during service construction. +func (g *DDLCommitGate) SetFrontierPublisher(publisher func(context.Context, timestamp.Timestamp) error) { + g.frontierPublisher = publisher +} + +func (g *DDLCommitGate) PublishDDLFrontier(ctx context.Context, ts timestamp.Timestamp) error { + g.RecordDDLFrontier(ts) + if g.frontierPublisher == nil || ts.IsEmpty() { + return nil + } + return g.frontierPublisher(ctx, ts) +} + +func (g *DDLCommitGate) LatestDDLFrontier() timestamp.Timestamp { + if current := g.ddlFrontier.Load(); current != nil { + return *current + } + return timestamp.Timestamp{} +} + +func publicBackgroundDDLBarrierEnabled(serviceID string) bool { + value, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(DDLCommitGateRuntimeKey) + if !ok || value == nil { + return false + } + gate, ok := value.(*DDLCommitGate) + return ok && gate.PublicDDLEnabled() +} + +func publishDDLCommitFrontier(ctx context.Context, serviceID string, ts timestamp.Timestamp) error { + value, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(DDLCommitGateRuntimeKey) + if !ok || value == nil { + return nil + } + if gate, ok := value.(*DDLCommitGate); ok { + return gate.PublishDDLFrontier(ctx, ts) + } + return moerr.NewInternalError(ctx, "invalid DDL commit gate") +} + +func recordDDLCommitFrontier(serviceID string, ts timestamp.Timestamp) { + value, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(DDLCommitGateRuntimeKey) + if !ok || value == nil { + return + } + if gate, ok := value.(*DDLCommitGate); ok { + gate.RecordDDLFrontier(ts) + } +} + +func enterDDLCommitGate(ctx context.Context, serviceID string) (func(), error) { + value, ok := moruntime.ServiceRuntime(serviceID).GetGlobalVariables(DDLCommitGateRuntimeKey) + if !ok || value == nil { + return func() {}, nil + } + gate, ok := value.(*DDLCommitGate) + if !ok { + return nil, moerr.NewInternalError(ctx, "invalid DDL commit gate") + } + return gate.Enter(ctx) +} + +// EnablePublicDDL marks that public listeners are live. Background sessions +// created on behalf of a client must participate in the cross-CN barrier after +// this point; bootstrap background DDL remains local before it. +func (g *DDLCommitGate) EnablePublicDDL() { + g.mu.Lock() + g.publicDDL = true + g.mu.Unlock() +} + +func (g *DDLCommitGate) PublicDDLEnabled() bool { + g.mu.Lock() + defer g.mu.Unlock() + // Close rejects every later Enter, but a DDL transaction admitted before + // shutdown retains the eligibility it had when public listeners were live. + return g.publicDDL +} + +func (g *DDLCommitGate) Enter(ctx context.Context) (func(), error) { + for { + g.mu.Lock() + if g.closed { + g.mu.Unlock() + return nil, moerr.NewServiceUnavailableNoCtx("DDL commit gate is closed") + } + if !g.blocked { + g.active++ + g.mu.Unlock() + var once sync.Once + return func() { + once.Do(func() { + g.mu.Lock() + g.active-- + g.signalLocked() + g.mu.Unlock() + }) + }, nil + } + changed := g.changed + hook := g.enterBlockedHook + g.mu.Unlock() + if hook != nil { + hook() + } + + select { + case <-ctx.Done(): + return nil, context.Cause(ctx) + case <-changed: + } + } +} + +func (g *DDLCommitGate) BlockNew() error { + g.mu.Lock() + defer g.mu.Unlock() + if g.closed { + return moerr.NewServiceUnavailableNoCtx("DDL commit gate is closed") + } + if !g.blocked { + g.blocked = true + g.signalLocked() + } + return nil +} + +func (g *DDLCommitGate) WaitDrained(ctx context.Context) error { + g.mu.Lock() + for g.active > 0 { + changed := g.changed + g.mu.Unlock() + select { + case <-ctx.Done(): + return context.Cause(ctx) + case <-changed: + } + g.mu.Lock() + if g.closed { + g.mu.Unlock() + return moerr.NewServiceUnavailableNoCtx("DDL commit gate is closed") + } + } + g.mu.Unlock() + return nil +} + +func (g *DDLCommitGate) Block(ctx context.Context) error { + if err := g.BlockNew(); err != nil { + return err + } + return g.WaitDrained(ctx) +} + +func (g *DDLCommitGate) Unblock() { + g.mu.Lock() + if !g.closed && g.blocked { + g.blocked = false + g.signalLocked() + } + g.mu.Unlock() +} + +func (g *DDLCommitGate) Close() { + g.mu.Lock() + if !g.closed { + g.closed = true + g.blocked = true + g.signalLocked() + } + g.mu.Unlock() +} + +func (g *DDLCommitGate) signalLocked() { + close(g.changed) + g.changed = make(chan struct{}) +} diff --git a/pkg/frontend/ddl_commit_gate_test.go b/pkg/frontend/ddl_commit_gate_test.go new file mode 100644 index 0000000000000..d2a57edcf8eb4 --- /dev/null +++ b/pkg/frontend/ddl_commit_gate_test.go @@ -0,0 +1,98 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package frontend + +import ( + "context" + "testing" + + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" + "github.com/stretchr/testify/require" +) + +func TestDDLCommitGateLinearizesBlockAndCommit(t *testing.T) { + gate := NewDDLCommitGate() + releaseFirst, err := gate.Enter(context.Background()) + require.NoError(t, err) + + gate.mu.Lock() + blockStarted := gate.changed + gate.mu.Unlock() + blockDone := make(chan error, 1) + go func() { blockDone <- gate.Block(context.Background()) }() + <-blockStarted // Block has published blocked=true. + select { + case err := <-blockDone: + t.Fatalf("block returned before the admitted DDL left: %v", err) + default: + } + releaseFirst() + require.NoError(t, <-blockDone) + + enterDone := make(chan error, 1) + go func() { + release, enterErr := gate.Enter(context.Background()) + if enterErr == nil { + release() + } + enterDone <- enterErr + }() + gate.mu.Lock() + require.True(t, gate.blocked) + require.Zero(t, gate.active) + gate.mu.Unlock() + select { + case err := <-enterDone: + t.Fatalf("DDL entered while activation held the gate: %v", err) + default: + } + gate.Unblock() + require.NoError(t, <-enterDone) +} + +func TestDDLCommitGateTracksOnlyMonotonicDDLFrontier(t *testing.T) { + gate := NewDDLCommitGate() + older := timestamp.Timestamp{PhysicalTime: 100} + newer := timestamp.Timestamp{PhysicalTime: 200} + + require.True(t, gate.LatestDDLFrontier().IsEmpty()) + gate.RecordDDLFrontier(newer) + gate.RecordDDLFrontier(older) + require.Equal(t, newer, gate.LatestDDLFrontier()) +} + +func TestDDLCommitGateCancellationAndClose(t *testing.T) { + gate := NewDDLCommitGate() + gate.EnablePublicDDL() + release, err := gate.Enter(context.Background()) + require.NoError(t, err) + ctx, cancel := context.WithCancel(context.Background()) + cancel() + err = gate.Block(ctx) + require.ErrorIs(t, err, context.Canceled) + release() + require.NoError(t, gate.Block(context.Background()), + "a canceled activation must retain the producer fence for a safe retry") + _, err = gate.Enter(ctx) + require.ErrorIs(t, err, context.Canceled) + + gate.Close() + require.True(t, gate.PublicDDLEnabled(), + "an admitted public background DDL retains shutdown fan-out eligibility") + _, err = gate.Enter(context.Background()) + require.ErrorContains(t, err, "closed") + err = gate.Block(context.Background()) + require.ErrorContains(t, err, "closed") +} diff --git a/pkg/frontend/txn.go b/pkg/frontend/txn.go index 85c15c787dbe4..ce880093815f6 100644 --- a/pkg/frontend/txn.go +++ b/pkg/frontend/txn.go @@ -17,6 +17,8 @@ package frontend import ( "context" "errors" + "maps" + "sort" "strings" "sync" "time" @@ -24,9 +26,13 @@ import ( "github.com/google/uuid" "go.uber.org/zap" + "github.com/matrixorigin/matrixone/pkg/clusterservice" "github.com/matrixorigin/matrixone/pkg/common/moerr" moruntime "github.com/matrixorigin/matrixone/pkg/common/runtime" "github.com/matrixorigin/matrixone/pkg/defines" + "github.com/matrixorigin/matrixone/pkg/pb/metadata" + querypb "github.com/matrixorigin/matrixone/pkg/pb/query" + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" pbtxn "github.com/matrixorigin/matrixone/pkg/pb/txn" "github.com/matrixorigin/matrixone/pkg/sql/parsers/tree" txnclient "github.com/matrixorigin/matrixone/pkg/txn/client" @@ -968,9 +974,23 @@ func (th *TxnHandler) commitUnsafe(execCtx *ExecCtx) error { if th.txnOp != nil { execCtx.ses.EnterFPrint(FPCommitUnsafeBeforeCommitWithTxn) defer execCtx.ses.ExitFPrint(FPCommitUnsafeBeforeCommitWithTxn) - commitTs := th.txnOp.Txn().CommitTS + txnMeta := th.txnOp.Txn() + commitTs := txnMeta.CommitTS + visibilityTS := timestamp.Timestamp{} + if !txnMeta.SnapshotTS.IsEmpty() { + // SnapshotTS is exclusive. A no-op DDL observed catalog state through + // SnapshotTS.Prev(), which is the frontier other CNs must also apply. + visibilityTS = txnMeta.SnapshotTS.Prev() + } tempTxnKey := tempTableTxnKey(th.txnOp) haveDDL := th.txnOp.GetWorkspace().GetHaveDDL() + if haveDDL { + releaseDDLCommit, gateErr := enterDDLCommitGate(ctx2, execCtx.ses.GetService()) + if gateErr != nil { + return gateErr + } + defer releaseDDLCommit() + } execCtx.ses.SetTxnId(th.txnOp.Txn().ID) commitResultUnknown := false err, hasRecovered = ExecuteFuncWithRecover(func() error { @@ -983,6 +1003,9 @@ func (th *TxnHandler) commitUnsafe(execCtx *ExecCtx) error { // left in a state that is unsafe to inspect. if !hasRecovered { commitTs = th.txnOp.Txn().CommitTS + if !commitTs.IsEmpty() { + visibilityTS = commitTs + } } if err != nil { err = moerr.AttachCause(ctx2, err) @@ -1014,6 +1037,21 @@ func (th *TxnHandler) commitUnsafe(execCtx *ExecCtx) error { owner.rollbackTempTableTransaction(tempTxnKey) } } + if haveDDL && !commitTs.IsEmpty() && (err == nil || commitResultUnknown) { + recordDDLCommitFrontier(execCtx.ses.GetService(), commitTs) + if execCtx.ses.GetFromRealUser() || + publicBackgroundDDLBarrierEnabled(execCtx.ses.GetService()) { + // A periodic heartbeat is insufficient here: acknowledging the DDL + // first leaves a crash window in which replacement loses T forever. + // Publish T through the HAKeeper RSM before returning success. For an + // unknown commit outcome, make the same best-effort durable advance. + if publishErr := publishDDLCommitFrontier( + ctx2, execCtx.ses.GetService(), commitTs, + ); publishErr != nil { + err = errors.Join(err, publishErr) + } + } + } if commitResultUnknown { // ErrTxnUnknown is terminal for this frontend handle. The operator // has already finalized its workspace non-destructively; retaining it @@ -1022,12 +1060,195 @@ func (th *TxnHandler) commitUnsafe(execCtx *ExecCtx) error { execCtx.ses.SetTxnId(dumpUUID[:]) return err } + if err == nil && haveDDL && !visibilityTS.IsEmpty() && + (execCtx.ses.GetFromRealUser() || + publicBackgroundDDLBarrierEnabled(execCtx.ses.GetService())) { + // A fresh proxy connection has no session commit timestamp to carry + // across CNs. Do not acknowledge a client DDL until every working CN + // has reached the commit, or the observed catalog frontier for a no-op. + err = syncDDLCommitToBarrierReadyCNs(ctx2, execCtx.ses.GetService(), visibilityTS) + } } th.invalidateTxnUnsafe() execCtx.ses.SetTxnId(dumpUUID[:]) return err } +type ddlVisibilityTarget struct { + serviceID string + generation uint64 + address string +} + +// syncDDLCommitToBarrierReadyCNs closes both sides of CN admission. A CN publishes +// barrier readiness before its startup frontier fence and public ingress. The +// DDL sender refreshes membership again after fan-out and repeats if any ready +// generation changed. A failed target is retried only after authoritative +// revalidation proves that exact generation/address tuple has departed. +// Protocol v44 is a deployment gate: mixed-version +// clusters retain legacy behavior without invoking an old receiver's fatal +// SyncCommit path. +func syncDDLCommitToBarrierReadyCNs( + ctx context.Context, + service string, + visibilityTS timestamp.Timestamp, +) error { + pu := getPuIfPresent(service) + if pu == nil || pu.QueryClient == nil { + return nil + } + + qc := pu.QueryClient + protocol, ok := moruntime.ServiceRuntime(qc.ServiceID()).GetGlobalVariables(moruntime.MOProtocolVersion) + protocolVersion, valid := protocol.(int64) + if !ok || !valid || protocolVersion < defines.MORPCVersion44 { + return nil + } + cluster := clusterservice.GetMOCluster(qc.ServiceID()) + if cluster == nil { + return moerr.NewInternalError(ctx, "cannot establish DDL visibility barrier: cluster service is unavailable") + } + refresher, ok := cluster.(clusterservice.AuthoritativeRefresher) + if !ok { + return moerr.NewInternalError(ctx, "cannot establish DDL visibility barrier: authoritative refresh is unavailable") + } + + for { + if err := refresher.Refresh(ctx); err != nil { + return errors.Join( + err, + moerr.NewInternalError(ctx, "failed to refresh CNs for DDL visibility barrier"), + ) + } + targets, err := ddlVisibilityTargets(ctx, cluster) + if err != nil { + return err + } + if len(targets) == 0 { + return moerr.NewInternalError(ctx, "cannot establish DDL visibility barrier: no barrier-ready CN is available") + } + + ordered := make([]ddlVisibilityTarget, 0, len(targets)) + for _, target := range targets { + ordered = append(ordered, target) + } + sort.Slice(ordered, func(i, j int) bool { return ordered[i].serviceID < ordered[j].serviceID }) + retryFanout := false + for _, target := range ordered { + req := qc.NewRequest(querypb.CmdMethod_SyncCommitV2) + req.SycnCommit = &querypb.SyncCommitRequest{LatestCommitTS: visibilityTS} + resp, sendErr := qc.SendMessage(ctx, target.address, req) + if sendErr != nil { + failure := errors.Join( + sendErr, + moerr.NewInternalErrorf(ctx, "failed to apply DDL commit on CN %s", target.address), + ) + retryFanout, err = revalidateDDLVisibilityTargetFailure( + ctx, refresher, cluster, target, failure) + if err != nil { + return err + } + break + } + if resp == nil { + failure := moerr.NewInternalErrorf( + ctx, "empty DDL visibility response from CN %s", target.address) + retryFanout, err = revalidateDDLVisibilityTargetFailure( + ctx, refresher, cluster, target, failure) + if err != nil { + return err + } + break + } + qc.Release(resp) + } + if retryFanout { + continue + } + + if err := refresher.Refresh(ctx); err != nil { + return errors.Join( + err, + moerr.NewInternalError(ctx, "failed to verify CNs for DDL visibility barrier"), + ) + } + verified, err := ddlVisibilityTargets(ctx, cluster) + if err != nil { + return err + } + if maps.Equal(targets, verified) { + return nil + } + } +} + +// revalidateDDLVisibilityTargetFailure distinguishes a graceful target +// withdrawal/replacement from a failure of a still-required barrier member. +// Unknown membership never becomes success: refresh and inventory errors are +// joined with the original target failure. +func revalidateDDLVisibilityTargetFailure( + ctx context.Context, + refresher clusterservice.AuthoritativeRefresher, + cluster clusterservice.MOCluster, + failed ddlVisibilityTarget, + failure error, +) (bool, error) { + if err := refresher.Refresh(ctx); err != nil { + return false, errors.Join( + failure, + err, + moerr.NewInternalError(ctx, "failed to revalidate CN after DDL visibility target failure"), + ) + } + targets, err := ddlVisibilityTargets(ctx, cluster) + if err != nil { + return false, errors.Join(failure, err) + } + current, ok := targets[failed.serviceID] + if ok && current == failed { + return false, failure + } + return true, nil +} + +func ddlVisibilityTargets( + ctx context.Context, + cluster clusterservice.MOCluster, +) (map[string]ddlVisibilityTarget, error) { + targets := make(map[string]ddlVisibilityTarget) + missingAddress := "" + err := clusterservice.GetCNServiceRawWithContext( + ctx, + cluster, + clusterservice.NewSelector(), + func(cn metadata.CNService) bool { + if !cn.DDLVisibilityBarrierReady { + return true + } + if cn.QueryAddress == "" { + missingAddress = cn.ServiceID + return false + } + targets[cn.ServiceID] = ddlVisibilityTarget{ + serviceID: cn.ServiceID, + generation: cn.ViewMetadataAdmissionGeneration, + address: cn.QueryAddress, + } + return true + }) + if err != nil { + return nil, err + } + if missingAddress != "" { + return nil, moerr.NewInternalErrorf( + ctx, + "cannot establish DDL visibility barrier: barrier-ready CN %s has no query address", + missingAddress, + ) + } + return targets, nil +} + // Rollback rolls back the txn // the option bits decide the actual behavior func (th *TxnHandler) Rollback(execCtx *ExecCtx) error { diff --git a/pkg/frontend/txn_test.go b/pkg/frontend/txn_test.go index 4002883302703..d8a19cd503ad2 100644 --- a/pkg/frontend/txn_test.go +++ b/pkg/frontend/txn_test.go @@ -28,12 +28,15 @@ import ( "github.com/smartystreets/goconvey/convey" "github.com/stretchr/testify/require" + "github.com/matrixorigin/matrixone/pkg/clusterservice" "github.com/matrixorigin/matrixone/pkg/common/moerr" moruntime "github.com/matrixorigin/matrixone/pkg/common/runtime" "github.com/matrixorigin/matrixone/pkg/defines" "github.com/matrixorigin/matrixone/pkg/frontend/databranchutils" mock_frontend "github.com/matrixorigin/matrixone/pkg/frontend/test" "github.com/matrixorigin/matrixone/pkg/pb/lock" + "github.com/matrixorigin/matrixone/pkg/pb/metadata" + querypb "github.com/matrixorigin/matrixone/pkg/pb/query" "github.com/matrixorigin/matrixone/pkg/pb/timestamp" "github.com/matrixorigin/matrixone/pkg/pb/txn" "github.com/matrixorigin/matrixone/pkg/sql/parsers/tree" @@ -856,6 +859,524 @@ func TestCommitUsesFinalCommitTSForNextTxn(t *testing.T) { } } +type ddlSyncRequest struct { + address string + method querypb.CmdMethod + commitTS timestamp.Timestamp +} + +type ddlSyncQueryClient struct { + serviceID string + sendError map[string]error + nilResponse map[string]bool + requests []ddlSyncRequest + releases int + beforeSend func() +} + +func (c *ddlSyncQueryClient) ServiceID() string { + return c.serviceID +} + +func (c *ddlSyncQueryClient) SendMessage( + _ context.Context, + address string, + req *querypb.Request, +) (*querypb.Response, error) { + request := ddlSyncRequest{address: address, method: req.CmdMethod} + if req.SycnCommit != nil { + request.commitTS = req.SycnCommit.LatestCommitTS + } + if c.beforeSend != nil { + c.beforeSend() + } + c.requests = append(c.requests, request) + if err := c.sendError[address]; err != nil { + return nil, err + } + if c.nilResponse[address] { + return nil, nil + } + return &querypb.Response{}, nil +} + +func (c *ddlSyncQueryClient) NewRequest(method querypb.CmdMethod) *querypb.Request { + return &querypb.Request{CmdMethod: method} +} + +func (c *ddlSyncQueryClient) Release(*querypb.Response) { + c.releases++ +} + +func (c *ddlSyncQueryClient) Close() error { + return nil +} + +func TestCommitSyncsDDLCommitToBarrierReadyCNs(t *testing.T) { + const queryServiceID = "ddl-sync-commit-test" + commitTS := timestamp.Timestamp{PhysicalTime: 100, LogicalTime: 7} + snapshotTS := timestamp.Timestamp{PhysicalTime: 90, LogicalTime: 5} + targets := []metadata.CNService{ + { + ServiceID: "cn-1", QueryAddress: "cn-1:6001", + ViewMetadataAdmissionGeneration: 1, DDLVisibilityBarrierReady: true, + }, + { + ServiceID: "cn-2", QueryAddress: "cn-2:6001", + ViewMetadataAdmissionGeneration: 1, DDLVisibilityBarrierReady: true, + }, + } + + newState := func(t *testing.T) (*Session, *testTxnOp, *ddlSyncQueryClient, *ExecCtx) { + t.Helper() + ctrl := gomock.NewController(t) + ctx := defines.AttachAccountId(context.Background(), sysAccountID) + ses := newTestSession(t, ctrl) + ses.SetFromRealUser(true) + eng := mock_frontend.NewMockEngine(ctrl) + eng.EXPECT().Hints().Return(engine.Hints{CommitOrRollbackTimeout: time.Second}).AnyTimes() + ses.txnHandler.storage = eng + + op := newTestTxnOp() + op.meta = txn.TxnMeta{ + ID: []byte{1, 2, 3, 4}, + Status: txn.TxnStatus_Active, + SnapshotTS: snapshotTS, + } + op.commitTS = commitTS + op.wp.SetHaveDDL(true) + ses.txnHandler.txnOp = op + ses.txnHandler.txnCtx = ctx + + qc := &ddlSyncQueryClient{ + serviceID: queryServiceID, + sendError: make(map[string]error), + nilResponse: make(map[string]bool), + } + pu := getPu("") + previousQueryClient := pu.QueryClient + pu.QueryClient = qc + t.Cleanup(func() { pu.QueryClient = previousQueryClient }) + moruntime.SetupServiceBasedRuntime(queryServiceID, moruntime.DefaultRuntime()) + moruntime.ServiceRuntime(queryServiceID).SetGlobalVariables( + moruntime.ClusterService, + &mockMOCluster{cnServices: targets}, + ) + + execCtx := newTestExecCtx(ctx, ctrl) + execCtx.ses = ses + execCtx.stmt = &tree.CreateTable{} + execCtx.txnOpt = FeTxnOption{autoCommit: true} + return ses, op, qc, execCtx + } + + t.Run("waits for every stable barrier-ready CN", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + }, qc.requests) + require.Equal(t, 2, qc.releases) + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + require.Equal(t, 2, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Zero(t, op.rollbackCalls) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("publishes frontier durably before acknowledgement and fan-out", func(t *testing.T) { + ses, _, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + gate := NewDDLCommitGate() + published := false + gate.SetFrontierPublisher(func(_ context.Context, ts timestamp.Timestamp) error { + require.Equal(t, commitTS, ts) + published = true + return nil + }) + qc.beforeSend = func() { + require.True(t, published, "HAKeeper frontier publication must precede fan-out") + } + rt := moruntime.ServiceRuntime(ses.GetService()) + previousGate, hadPreviousGate := rt.GetGlobalVariables(DDLCommitGateRuntimeKey) + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, gate) + t.Cleanup(func() { + if hadPreviousGate { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, previousGate) + } else { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, nil) + } + }) + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.True(t, published) + require.Equal(t, commitTS, gate.LatestDDLFrontier()) + }) + + t.Run("publication failure prevents successful acknowledgement", func(t *testing.T) { + ses, _, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + publishErr := errors.New("injected durable frontier publication failure") + gate := NewDDLCommitGate() + gate.SetFrontierPublisher(func(context.Context, timestamp.Timestamp) error { return publishErr }) + rt := moruntime.ServiceRuntime(ses.GetService()) + previousGate, hadPreviousGate := rt.GetGlobalVariables(DDLCommitGateRuntimeKey) + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, gate) + t.Cleanup(func() { + if hadPreviousGate { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, previousGate) + } else { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, nil) + } + }) + + require.ErrorIs(t, ses.GetTxnHandler().Commit(execCtx), publishErr) + require.Empty(t, qc.requests, "fan-out and success must not follow failed durable publication") + }) + + t.Run("activation gate drains and blocks concurrent DDL commit", func(t *testing.T) { + ses, op, _, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + gate := NewDDLCommitGate() + require.NoError(t, gate.Block(context.Background())) + var once sync.Once + commitWaiting := make(chan struct{}) + gate.enterBlockedHook = func() { once.Do(func() { close(commitWaiting) }) } + rt := moruntime.ServiceRuntime(ses.GetService()) + previousGate, hadPreviousGate := rt.GetGlobalVariables(DDLCommitGateRuntimeKey) + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, gate) + t.Cleanup(func() { + if hadPreviousGate { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, previousGate) + } else { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, nil) + } + }) + + commitDone := make(chan error, 1) + go func() { commitDone <- ses.GetTxnHandler().Commit(execCtx) }() + <-commitWaiting + require.Zero(t, op.commitCalls, + "DDL commit must not cross the activation linearization point") + gate.Unblock() + require.NoError(t, <-commitDone) + require.Equal(t, 1, op.commitCalls) + }) + + t.Run("retries when a CN becomes barrier-ready during fan-out", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + expanded := append([]metadata.CNService{}, targets...) + expanded = append(expanded, metadata.CNService{ + ServiceID: "cn-3", QueryAddress: "cn-3:6001", + ViewMetadataAdmissionGeneration: 7, DDLVisibilityBarrierReady: true, + }) + cluster.refreshSnapshots = [][]metadata.CNService{targets, expanded, expanded, expanded} + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-3:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + }, qc.requests) + require.Equal(t, 5, qc.releases) + require.Equal(t, 4, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("retries when a barrier-ready CN generation is replaced", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + replaced := append([]metadata.CNService{}, targets...) + replaced[1].ViewMetadataAdmissionGeneration = 2 + replaced[1].QueryAddress = "cn-2-new:6001" + cluster.refreshSnapshots = [][]metadata.CNService{targets, replaced, replaced, replaced} + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2-new:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + }, qc.requests) + require.Equal(t, 4, qc.releases) + require.Equal(t, 4, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("retries when a failed target withdrew after the membership snapshot", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + sendErr := moerr.NewInternalErrorNoCtx("closing CN unavailable") + qc.sendError["cn-2:6001"] = sendErr + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + remaining := append([]metadata.CNService{}, targets[:1]...) + cluster.refreshSnapshots = [][]metadata.CNService{targets, remaining, remaining, remaining} + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + }, qc.requests) + require.Equal(t, 2, qc.releases) + require.Equal(t, 4, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Zero(t, op.rollbackCalls) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("retries when a failed target tuple is replaced", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + sendErr := moerr.NewInternalErrorNoCtx("replaced CN unavailable") + qc.sendError["cn-2:6001"] = sendErr + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + replaced := append([]metadata.CNService{}, targets...) + replaced[1].ViewMetadataAdmissionGeneration = 2 + replaced[1].QueryAddress = "cn-2-new:6001" + cluster.refreshSnapshots = [][]metadata.CNService{targets, replaced, replaced, replaced} + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2-new:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + }, qc.requests) + require.Equal(t, 3, qc.releases) + require.Equal(t, 4, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Zero(t, op.rollbackCalls) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("mixed-version cluster does not call a legacy receiver", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + moruntime.ServiceRuntime(queryServiceID).SetGlobalVariables( + moruntime.MOProtocolVersion, + defines.MORPCVersion34, + ) + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Empty(t, qc.requests) + require.Zero(t, qc.releases) + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + require.Zero(t, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("does not acknowledge a membership refresh failure", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + refreshErr := moerr.NewInternalErrorNoCtx("HAKeeper unavailable") + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + cluster.refreshError = refreshErr + + err := ses.GetTxnHandler().Commit(execCtx) + require.ErrorIs(t, err, refreshErr) + require.Equal(t, 1, cluster.refreshCalls) + require.Empty(t, qc.requests) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Zero(t, op.rollbackCalls) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("does not acknowledge a partial barrier", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + sendErr := moerr.NewInternalErrorNoCtx("CN unavailable") + qc.sendError["cn-2:6001"] = sendErr + + err := ses.GetTxnHandler().Commit(execCtx) + require.ErrorIs(t, err, sendErr) + require.Len(t, qc.requests, 2) + require.Equal(t, 1, qc.releases) + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + require.Equal(t, 2, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Zero(t, op.rollbackCalls) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("does not retry when failure revalidation cannot refresh membership", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + sendErr := moerr.NewInternalErrorNoCtx("CN unavailable") + refreshErr := moerr.NewInternalErrorNoCtx("HAKeeper unavailable after send failure") + qc.sendError["cn-2:6001"] = sendErr + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + cluster.refreshErrors = map[int]error{2: refreshErr} + + err := ses.GetTxnHandler().Commit(execCtx) + require.ErrorIs(t, err, sendErr) + require.ErrorIs(t, err, refreshErr) + require.Len(t, qc.requests, 2) + require.Equal(t, 1, qc.releases) + require.Equal(t, 2, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Zero(t, op.rollbackCalls) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("rejects an empty CN response", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + qc.nilResponse["cn-1:6001"] = true + + err := ses.GetTxnHandler().Commit(execCtx) + require.ErrorContains(t, err, "empty DDL visibility response") + require.Len(t, qc.requests, 1) + require.Zero(t, qc.releases) + cluster := clusterservice.GetMOCluster(queryServiceID).(*mockMOCluster) + require.Equal(t, 2, cluster.refreshCalls) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("rejects an empty barrier-ready CN set", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + moruntime.ServiceRuntime(queryServiceID).SetGlobalVariables( + moruntime.ClusterService, + &mockMOCluster{}, + ) + + err := ses.GetTxnHandler().Commit(execCtx) + require.ErrorContains(t, err, "no barrier-ready CN is available") + require.Empty(t, qc.requests) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("rejects a barrier-ready CN without query address", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + moruntime.ServiceRuntime(queryServiceID).SetGlobalVariables( + moruntime.ClusterService, + &mockMOCluster{cnServices: []metadata.CNService{{ + ServiceID: "cn-missing-address", DDLVisibilityBarrierReady: true, + }}}, + ) + + err := ses.GetTxnHandler().Commit(execCtx) + require.ErrorContains(t, err, "barrier-ready CN cn-missing-address has no query address") + require.Empty(t, qc.requests) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("synchronizes the observed frontier for a DDL no-op", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + op.commitTS = timestamp.Timestamp{} + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: snapshotTS.Prev()}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: snapshotTS.Prev()}, + }, qc.requests) + require.Equal(t, 2, qc.releases) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + require.Nil(t, ses.GetTxnHandler().GetTxn()) + }) + + t.Run("live client-owned background DDL participates in barrier", func(t *testing.T) { + ses, op, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + ses.SetFromRealUser(false) + gate := NewDDLCommitGate() + gate.EnablePublicDDL() + rt := moruntime.ServiceRuntime(ses.GetService()) + previousGate, hadPreviousGate := rt.GetGlobalVariables(DDLCommitGateRuntimeKey) + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, gate) + t.Cleanup(func() { + if hadPreviousGate { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, previousGate) + } else { + rt.SetGlobalVariables(DDLCommitGateRuntimeKey, nil) + } + }) + + commitEntered := make(chan struct{}) + allowCommit := make(chan struct{}) + op.commitHook = func() { + close(commitEntered) + <-allowCommit + } + commitDone := make(chan error, 1) + go func() { commitDone <- ses.GetTxnHandler().Commit(execCtx) }() + <-commitEntered + gate.Close() + close(allowCommit) + require.NoError(t, <-commitDone) + require.Equal(t, []ddlSyncRequest{ + {address: "cn-1:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + {address: "cn-2:6001", method: querypb.CmdMethod_SyncCommitV2, commitTS: commitTS}, + }, qc.requests) + require.Equal(t, 2, qc.releases) + require.Equal(t, txn.TxnStatus_Committed, op.meta.Status) + }) + + tests := []struct { + name string + configure func(*Session, *testTxnOp) + }{ + { + name: "non-DDL transaction", + configure: func(_ *Session, op *testTxnOp) { + op.wp.SetHaveDDL(false) + }, + }, + { + name: "pre-ingress internal session", + configure: func(ses *Session, _ *testTxnOp) { + ses.SetFromRealUser(false) + }, + }, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + ses, _, qc, execCtx := newState(t) + defer ses.Close() + defer execCtx.Close() + test.configure(ses, ses.txnHandler.txnOp.(*testTxnOp)) + + require.NoError(t, ses.GetTxnHandler().Commit(execCtx)) + require.Empty(t, qc.requests) + require.Zero(t, qc.releases) + }) + } +} + func TestCreateUnsafeRollsBackPublishedGenerationOnPanic(t *testing.T) { ctrl := gomock.NewController(t) ctx := defines.AttachAccountId(context.Background(), sysAccountID) diff --git a/pkg/hakeeper/rsm.go b/pkg/hakeeper/rsm.go index fa9e52ef76526..5c685d6c998fc 100644 --- a/pkg/hakeeper/rsm.go +++ b/pkg/hakeeper/rsm.go @@ -1705,10 +1705,12 @@ func (s *stateMachine) handleScheduleCommandQuery(uuid string) *pb.CommandBatch func (s *stateMachine) handleClusterDetailsQuery(cfg Config) *pb.ClusterDetails { cfg.Fill() cd := &pb.ClusterDetails{ - CNStores: make([]pb.CNStore, 0, len(s.state.CNState.Stores)), - TNStores: make([]pb.TNStore, 0, len(s.state.TNState.Stores)), - LogStores: make([]pb.LogStore, 0, len(s.state.LogState.Stores)), - ProxyStores: make([]pb.ProxyStore, 0, len(s.state.ProxyState.Stores)), + CNStores: make([]pb.CNStore, 0, len(s.state.CNState.Stores)), + TNStores: make([]pb.TNStore, 0, len(s.state.TNState.Stores)), + LogStores: make([]pb.LogStore, 0, len(s.state.LogState.Stores)), + ProxyStores: make([]pb.ProxyStore, 0, len(s.state.ProxyState.Stores)), + DDLVisibilityDeployedProtocol: s.state.CNState.DDLVisibilityDeployedProtocol, + DDLVisibilityFrontier: s.state.CNState.DDLVisibilityFrontier, } if s.viewMetadataAdmissionActive() { cd.ViewMetadataAdmission = &pb.ViewMetadataAdmission{ @@ -1749,6 +1751,12 @@ func (s *stateMachine) handleClusterDetailsQuery(cfg Config) *pb.ClusterDetails ViewMetadataAdmissionGeneration: info.ViewMetadataAdmissionGeneration, ViewMetadataObservedEpoch: info.ViewMetadataObservedEpoch, ViewMetadataAdmissionReady: info.ViewMetadataAdmissionReady, + DDLVisibilityBarrierReady: info.DDLVisibilityBarrierReady, + DDLVisibilityDeployedProtocol: info.DDLVisibilityDeployedProtocol, + DDLVisibilityActivationPrepared: info.DDLVisibilityActivationPrepared, + DDLVisibilityActivationFenced: info.DDLVisibilityActivationFenced, + DDLVisibilityFrontier: info.DDLVisibilityFrontier, + ViewMetadataIngressReady: info.ViewMetadataIngressReady, } cd.CNStores = append(cd.CNStores, n) } @@ -1778,13 +1786,14 @@ func (s *stateMachine) handleClusterDetailsQuery(cfg Config) *pb.ClusterDetails state = pb.TimeoutState } n := pb.LogStore{ - UUID: uuid, - Tick: info.Tick, - State: state, - ServiceAddress: info.ServiceAddress, - Replicas: info.Replicas, - ConfigData: info.ConfigData, - Locality: info.Locality, + UUID: uuid, + Tick: info.Tick, + State: state, + ServiceAddress: info.ServiceAddress, + Replicas: info.Replicas, + ConfigData: info.ConfigData, + Locality: info.Locality, + DDLVisibilityDeployedProtocolSupported: info.DDLVisibilityDeployedProtocolSupported, } cd.LogStores = append(cd.LogStores, n) } diff --git a/pkg/hakeeper/view_metadata_admission.go b/pkg/hakeeper/view_metadata_admission.go index f447aabbd3cd4..75770a0141417 100644 --- a/pkg/hakeeper/view_metadata_admission.go +++ b/pkg/hakeeper/view_metadata_admission.go @@ -605,6 +605,11 @@ func (s *stateMachine) attachViewMetadataAdmission( } } batch.ViewMetadataAdmission = s.viewMetadataAdmissionSnapshot(uuid, proxy) + if !proxy { + batch.DDLVisibilityDeployedProtocol = s.state.CNState.DDLVisibilityDeployedProtocol + frontier := s.state.CNState.DDLVisibilityFrontier + batch.DDLVisibilityFrontier = &frontier + } data, err := batch.Marshal() if err != nil { panic(err) diff --git a/pkg/hakeeper/view_metadata_admission_test.go b/pkg/hakeeper/view_metadata_admission_test.go index 4ecd65439a874..9ac66e2148b17 100644 --- a/pkg/hakeeper/view_metadata_admission_test.go +++ b/pkg/hakeeper/view_metadata_admission_test.go @@ -653,6 +653,7 @@ func TestClusterDetailsKeepsSupportedPendingButHidesLegacyIngress(t *testing.T) rsm.state.CNState.Stores["pending"] = pb.CNStoreInfo{ ViewMetadataAdmissionSupported: true, ViewMetadataAdmissionGeneration: 2, + DDLVisibilityBarrierReady: true, } rsm.state.CNState.Stores["ready"] = pb.CNStoreInfo{ ViewMetadataAdmissionSupported: true, @@ -670,7 +671,9 @@ func TestClusterDetailsKeepsSupportedPendingButHidesLegacyIngress(t *testing.T) } require.Contains(t, byID, "pending") require.False(t, byID["pending"].ViewMetadataAdmissionReady) + require.True(t, byID["pending"].DDLVisibilityBarrierReady) require.Contains(t, byID, "ready") + require.True(t, byID["ready"].ViewMetadataIngressReady) require.NotContains(t, byID, "legacy") } diff --git a/pkg/logservice/store.go b/pkg/logservice/store.go index 10eaa52caddc5..796b54c54b4c3 100644 --- a/pkg/logservice/store.go +++ b/pkg/logservice/store.go @@ -1594,15 +1594,16 @@ func (l *store) hakeeperTick() { func (l *store) getHeartbeatMessage() pb.LogStoreHeartbeat { m := pb.LogStoreHeartbeat{ - UUID: l.id(), - RaftAddress: l.cfg.RaftServiceAddr(), - ServiceAddress: l.cfg.LogServiceServiceAddr(), - GossipAddress: l.cfg.GossipServiceAddr(), - StoreIncarnation: l.getStoreIncarnation(), - Replicas: make([]pb.LogReplicaInfo, 0), - Locality: l.cfg.getLocality(), - CommandDeliverySupported: true, - ViewMetadataAdmissionSupported: true, + UUID: l.id(), + RaftAddress: l.cfg.RaftServiceAddr(), + ServiceAddress: l.cfg.LogServiceServiceAddr(), + GossipAddress: l.cfg.GossipServiceAddr(), + StoreIncarnation: l.getStoreIncarnation(), + Replicas: make([]pb.LogReplicaInfo, 0), + Locality: l.cfg.getLocality(), + CommandDeliverySupported: true, + ViewMetadataAdmissionSupported: true, + DDLVisibilityDeployedProtocolSupported: true, } opts := dragonboat.NodeHostInfoOption{ SkipLogInfo: true, diff --git a/pkg/pb/logservice/logservice.go b/pkg/pb/logservice/logservice.go index 8d6176479d8ac..27b8a0a7a0f43 100644 --- a/pkg/pb/logservice/logservice.go +++ b/pkg/pb/logservice/logservice.go @@ -163,7 +163,53 @@ func (s *CNState) Update(hb CNStoreHeartbeat, tick uint64) { storeInfo.ViewMetadataRefreshSupported = hb.ViewMetadataRefreshSupported storeInfo.ViewMetadataRevalidatedEpoch = hb.ViewMetadataRevalidatedEpoch storeInfo.ViewMetadataIngressReady = hb.ViewMetadataIngressReady + if s.DDLVisibilityDeployedProtocol > 0 && + hb.DDLVisibilityDeployedProtocol < s.DDLVisibilityDeployedProtocol { + // Linearize markerless joins with the committed cut. A stale producer may + // remain barrier-reachable, but it cannot become publicly routable. + storeInfo.ViewMetadataIngressReady = false + } + storeInfo.DDLVisibilityBarrierReady = hb.DDLVisibilityBarrierReady + storeInfo.DDLVisibilityDeployedProtocol = hb.DDLVisibilityDeployedProtocol + storeInfo.DDLVisibilityActivationPrepared = hb.DDLVisibilityActivationPrepared + storeInfo.DDLVisibilityActivationFenced = hb.DDLVisibilityActivationFenced + storeInfo.DDLVisibilityFrontier = hb.DDLVisibilityFrontier s.Stores[hb.UUID] = storeInfo + if s.DDLVisibilityFrontier.Less(hb.DDLVisibilityFrontier) { + s.DDLVisibilityFrontier = hb.DDLVisibilityFrontier + } + if hb.DDLVisibilityDeployedProtocol > s.DDLVisibilityDeployedProtocol && + len(hb.DDLVisibilityEpochCommitTargets) > 0 && + s.matchesDDLVisibilityEpochCommitTargets(hb.DDLVisibilityEpochCommitTargets) { + s.DDLVisibilityDeployedProtocol = hb.DDLVisibilityDeployedProtocol + } +} + +func (s *CNState) matchesDDLVisibilityEpochCommitTargets(targets []DDLVisibilityActivationTarget) bool { + expected := make(map[string]DDLVisibilityActivationTarget, len(targets)) + for _, target := range targets { + if target.ServiceID == "" || target.Generation == 0 || target.QueryAddress == "" { + return false + } + if _, ok := expected[target.ServiceID]; ok { + return false + } + expected[target.ServiceID] = target + } + seen := 0 + for serviceID, store := range s.Stores { + if store.QueryAddress == "" || store.ViewMetadataAdmissionGeneration == 0 { + continue + } + seen++ + target, ok := expected[serviceID] + if !ok || target.Generation != store.ViewMetadataAdmissionGeneration || + target.QueryAddress != store.QueryAddress || !store.DDLVisibilityBarrierReady || + !store.DDLVisibilityActivationPrepared || !store.DDLVisibilityActivationFenced { + return false + } + } + return seen == len(expected) } // UpdateLabel updates labels of CN store. @@ -274,6 +320,7 @@ func (s *LogState) updateStores(hb LogStoreHeartbeat, tick uint64) { storeInfo.Locality = hb.Locality storeInfo.CommandDeliverySupported = hb.CommandDeliverySupported storeInfo.ViewMetadataAdmissionSupported = hb.ViewMetadataAdmissionSupported + storeInfo.DDLVisibilityDeployedProtocolSupported = hb.DDLVisibilityDeployedProtocolSupported // Preserve a known incarnation when receiving a heartbeat from an older // binary during a rolling upgrade. Incarnation fencing is enabled only once // the store has reported a non-empty value. diff --git a/pkg/pb/logservice/logservice.pb.go b/pkg/pb/logservice/logservice.pb.go index f1bec4f1b5e27..77834f03282c8 100644 --- a/pkg/pb/logservice/logservice.pb.go +++ b/pkg/pb/logservice/logservice.pb.go @@ -15,6 +15,7 @@ import ( proto "github.com/gogo/protobuf/proto" github_com_gogo_protobuf_types "github.com/gogo/protobuf/types" metadata "github.com/matrixorigin/matrixone/pkg/pb/metadata" + timestamp "github.com/matrixorigin/matrixone/pkg/pb/timestamp" _ "google.golang.org/protobuf/types/known/timestamppb" ) @@ -530,9 +531,18 @@ type CNStore struct { ViewMetadataAdmissionGeneration uint64 `protobuf:"varint,18,opt,name=ViewMetadataAdmissionGeneration,proto3" json:"ViewMetadataAdmissionGeneration,omitempty"` ViewMetadataObservedEpoch uint64 `protobuf:"varint,19,opt,name=ViewMetadataObservedEpoch,proto3" json:"ViewMetadataObservedEpoch,omitempty"` ViewMetadataAdmissionReady bool `protobuf:"varint,20,opt,name=ViewMetadataAdmissionReady,proto3" json:"ViewMetadataAdmissionReady,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + // DDLVisibilityBarrierReady means QueryService can accept the versioned DDL + // visibility fence even when public SQL ingress is not admitted yet. + DDLVisibilityBarrierReady bool `protobuf:"varint,21,opt,name=DDLVisibilityBarrierReady,proto3" json:"DDLVisibilityBarrierReady,omitempty"` + // ViewMetadataIngressReady is retained for raw control-plane inventory. + ViewMetadataIngressReady bool `protobuf:"varint,22,opt,name=ViewMetadataIngressReady,proto3" json:"ViewMetadataIngressReady,omitempty"` + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,23,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + DDLVisibilityActivationPrepared bool `protobuf:"varint,24,opt,name=DDLVisibilityActivationPrepared,proto3" json:"DDLVisibilityActivationPrepared,omitempty"` + DDLVisibilityActivationFenced bool `protobuf:"varint,25,opt,name=DDLVisibilityActivationFenced,proto3" json:"DDLVisibilityActivationFenced,omitempty"` + DDLVisibilityFrontier timestamp.Timestamp `protobuf:"bytes,26,opt,name=DDLVisibilityFrontier,proto3" json:"DDLVisibilityFrontier"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CNStore) Reset() { *m = CNStore{} } @@ -701,6 +711,48 @@ func (m *CNStore) GetViewMetadataAdmissionReady() bool { return false } +func (m *CNStore) GetDDLVisibilityBarrierReady() bool { + if m != nil { + return m.DDLVisibilityBarrierReady + } + return false +} + +func (m *CNStore) GetViewMetadataIngressReady() bool { + if m != nil { + return m.ViewMetadataIngressReady + } + return false +} + +func (m *CNStore) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + +func (m *CNStore) GetDDLVisibilityActivationPrepared() bool { + if m != nil { + return m.DDLVisibilityActivationPrepared + } + return false +} + +func (m *CNStore) GetDDLVisibilityActivationFenced() bool { + if m != nil { + return m.DDLVisibilityActivationFenced + } + return false +} + +func (m *CNStore) GetDDLVisibilityFrontier() timestamp.Timestamp { + if m != nil { + return m.DDLVisibilityFrontier + } + return timestamp.Timestamp{} +} + type TNStore struct { UUID string `protobuf:"bytes,1,opt,name=UUID,proto3" json:"UUID,omitempty"` ServiceAddress string `protobuf:"bytes,2,opt,name=ServiceAddress,proto3" json:"ServiceAddress,omitempty"` @@ -832,16 +884,19 @@ func (m *TNStore) GetAutoIncrEpochFenceSupported() bool { } type LogStore struct { - UUID string `protobuf:"bytes,1,opt,name=UUID,proto3" json:"UUID,omitempty"` - ServiceAddress string `protobuf:"bytes,2,opt,name=ServiceAddress,proto3" json:"ServiceAddress,omitempty"` - Tick uint64 `protobuf:"varint,3,opt,name=Tick,proto3" json:"Tick,omitempty"` - State NodeState `protobuf:"varint,4,opt,name=State,proto3,enum=logservice.NodeState" json:"State,omitempty"` - Replicas []LogReplicaInfo `protobuf:"bytes,5,rep,name=Replicas,proto3" json:"Replicas"` - ConfigData *ConfigData `protobuf:"bytes,6,opt,name=ConfigData,proto3" json:"ConfigData,omitempty"` - Locality Locality `protobuf:"bytes,7,opt,name=Locality,proto3" json:"Locality"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + UUID string `protobuf:"bytes,1,opt,name=UUID,proto3" json:"UUID,omitempty"` + ServiceAddress string `protobuf:"bytes,2,opt,name=ServiceAddress,proto3" json:"ServiceAddress,omitempty"` + Tick uint64 `protobuf:"varint,3,opt,name=Tick,proto3" json:"Tick,omitempty"` + State NodeState `protobuf:"varint,4,opt,name=State,proto3,enum=logservice.NodeState" json:"State,omitempty"` + Replicas []LogReplicaInfo `protobuf:"bytes,5,rep,name=Replicas,proto3" json:"Replicas"` + ConfigData *ConfigData `protobuf:"bytes,6,opt,name=ConfigData,proto3" json:"ConfigData,omitempty"` + Locality Locality `protobuf:"bytes,7,opt,name=Locality,proto3" json:"Locality"` + // DDLVisibilityDeployedProtocolSupported proves the hosted HAKeeper RSM can + // persist and return the cluster-wide DDL deployment epoch. + DDLVisibilityDeployedProtocolSupported bool `protobuf:"varint,8,opt,name=DDLVisibilityDeployedProtocolSupported,proto3" json:"DDLVisibilityDeployedProtocolSupported,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *LogStore) Reset() { *m = LogStore{} } @@ -926,6 +981,13 @@ func (m *LogStore) GetLocality() Locality { return Locality{} } +func (m *LogStore) GetDDLVisibilityDeployedProtocolSupported() bool { + if m != nil { + return m.DDLVisibilityDeployedProtocolSupported + } + return false +} + // LogShardInfo contains information a log shard. type LogShardInfo struct { // ShardID is the ID of a Log shard. @@ -1167,6 +1229,69 @@ func (m *Resource) GetMemAvailable() uint64 { } // CNStoreHeartbeat is the periodic message sent tp the HAKeeper by CN stores. +type DDLVisibilityActivationTarget struct { + ServiceID string `protobuf:"bytes,1,opt,name=ServiceID,proto3" json:"ServiceID,omitempty"` + Generation uint64 `protobuf:"varint,2,opt,name=Generation,proto3" json:"Generation,omitempty"` + QueryAddress string `protobuf:"bytes,3,opt,name=QueryAddress,proto3" json:"QueryAddress,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` +} + +func (m *DDLVisibilityActivationTarget) Reset() { *m = DDLVisibilityActivationTarget{} } +func (m *DDLVisibilityActivationTarget) String() string { return proto.CompactTextString(m) } +func (*DDLVisibilityActivationTarget) ProtoMessage() {} +func (*DDLVisibilityActivationTarget) Descriptor() ([]byte, []int) { + return fileDescriptor_fd1040c5381ab5a7, []int{6} +} +func (m *DDLVisibilityActivationTarget) XXX_Unmarshal(b []byte) error { + return m.Unmarshal(b) +} +func (m *DDLVisibilityActivationTarget) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { + if deterministic { + return xxx_messageInfo_DDLVisibilityActivationTarget.Marshal(b, m, deterministic) + } else { + b = b[:cap(b)] + n, err := m.MarshalToSizedBuffer(b) + if err != nil { + return nil, err + } + return b[:n], nil + } +} +func (m *DDLVisibilityActivationTarget) XXX_Merge(src proto.Message) { + xxx_messageInfo_DDLVisibilityActivationTarget.Merge(m, src) +} +func (m *DDLVisibilityActivationTarget) XXX_Size() int { + return m.ProtoSize() +} +func (m *DDLVisibilityActivationTarget) XXX_DiscardUnknown() { + xxx_messageInfo_DDLVisibilityActivationTarget.DiscardUnknown(m) +} + +var xxx_messageInfo_DDLVisibilityActivationTarget proto.InternalMessageInfo + +func (m *DDLVisibilityActivationTarget) GetServiceID() string { + if m != nil { + return m.ServiceID + } + return "" +} + +func (m *DDLVisibilityActivationTarget) GetGeneration() uint64 { + if m != nil { + return m.Generation + } + return 0 +} + +func (m *DDLVisibilityActivationTarget) GetQueryAddress() string { + if m != nil { + return m.QueryAddress + } + return "" +} + type CNStoreHeartbeat struct { UUID string `protobuf:"bytes,1,opt,name=UUID,proto3" json:"UUID,omitempty"` ServiceAddress string `protobuf:"bytes,2,opt,name=ServiceAddress,proto3" json:"ServiceAddress,omitempty"` @@ -1196,17 +1321,27 @@ type CNStoreHeartbeat struct { ViewMetadataRevalidatedEpoch uint64 `protobuf:"varint,23,opt,name=ViewMetadataRevalidatedEpoch,proto3" json:"ViewMetadataRevalidatedEpoch,omitempty"` // ViewMetadataIngressReady is published only after every public CN ingress // listener has started successfully. Admission alone is not routability. - ViewMetadataIngressReady bool `protobuf:"varint,24,opt,name=ViewMetadataIngressReady,proto3" json:"ViewMetadataIngressReady,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + ViewMetadataIngressReady bool `protobuf:"varint,24,opt,name=ViewMetadataIngressReady,proto3" json:"ViewMetadataIngressReady,omitempty"` + DDLVisibilityBarrierReady bool `protobuf:"varint,25,opt,name=DDLVisibilityBarrierReady,proto3" json:"DDLVisibilityBarrierReady,omitempty"` + // DDLVisibilityDeployedProtocol reports only a locally committed deployment + // marker. Provisional activation must never advance the cluster-wide epoch. + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,26,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + // Non-empty only for the transition that atomically validates the complete + // CN membership tuple set and commits DDLVisibilityDeployedProtocol. + DDLVisibilityEpochCommitTargets []DDLVisibilityActivationTarget `protobuf:"bytes,27,rep,name=DDLVisibilityEpochCommitTargets,proto3" json:"DDLVisibilityEpochCommitTargets"` + DDLVisibilityActivationPrepared bool `protobuf:"varint,28,opt,name=DDLVisibilityActivationPrepared,proto3" json:"DDLVisibilityActivationPrepared,omitempty"` + DDLVisibilityActivationFenced bool `protobuf:"varint,29,opt,name=DDLVisibilityActivationFenced,proto3" json:"DDLVisibilityActivationFenced,omitempty"` + DDLVisibilityFrontier timestamp.Timestamp `protobuf:"bytes,30,opt,name=DDLVisibilityFrontier,proto3" json:"DDLVisibilityFrontier"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CNStoreHeartbeat) Reset() { *m = CNStoreHeartbeat{} } func (m *CNStoreHeartbeat) String() string { return proto.CompactTextString(m) } func (*CNStoreHeartbeat) ProtoMessage() {} func (*CNStoreHeartbeat) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{6} + return fileDescriptor_fd1040c5381ab5a7, []int{7} } func (m *CNStoreHeartbeat) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -1396,6 +1531,48 @@ func (m *CNStoreHeartbeat) GetViewMetadataIngressReady() bool { return false } +func (m *CNStoreHeartbeat) GetDDLVisibilityBarrierReady() bool { + if m != nil { + return m.DDLVisibilityBarrierReady + } + return false +} + +func (m *CNStoreHeartbeat) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + +func (m *CNStoreHeartbeat) GetDDLVisibilityEpochCommitTargets() []DDLVisibilityActivationTarget { + if m != nil { + return m.DDLVisibilityEpochCommitTargets + } + return nil +} + +func (m *CNStoreHeartbeat) GetDDLVisibilityActivationPrepared() bool { + if m != nil { + return m.DDLVisibilityActivationPrepared + } + return false +} + +func (m *CNStoreHeartbeat) GetDDLVisibilityActivationFenced() bool { + if m != nil { + return m.DDLVisibilityActivationFenced + } + return false +} + +func (m *CNStoreHeartbeat) GetDDLVisibilityFrontier() timestamp.Timestamp { + if m != nil { + return m.DDLVisibilityFrontier + } + return timestamp.Timestamp{} +} + // CNAllocateID is the periodic message sent tp the HAKeeper by CN stores. type CNAllocateID struct { Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty"` @@ -1412,7 +1589,7 @@ func (m *CNAllocateID) Reset() { *m = CNAllocateID{} } func (m *CNAllocateID) String() string { return proto.CompactTextString(m) } func (*CNAllocateID) ProtoMessage() {} func (*CNAllocateID) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{7} + return fileDescriptor_fd1040c5381ab5a7, []int{8} } func (m *CNAllocateID) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -1488,17 +1665,20 @@ type LogStoreHeartbeat struct { // StoreIncarnation identifies the persistent storage generation used by // this Log store. It remains stable across process restarts and changes when // the store starts from a new data directory. - StoreIncarnation string `protobuf:"bytes,11,opt,name=StoreIncarnation,proto3" json:"StoreIncarnation,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + StoreIncarnation string `protobuf:"bytes,11,opt,name=StoreIncarnation,proto3" json:"StoreIncarnation,omitempty"` + // This gates the HAKeeper RSM schema used by the durable DDL deployment + // epoch across both voting and non-voting replicas. + DDLVisibilityDeployedProtocolSupported bool `protobuf:"varint,12,opt,name=DDLVisibilityDeployedProtocolSupported,proto3" json:"DDLVisibilityDeployedProtocolSupported,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *LogStoreHeartbeat) Reset() { *m = LogStoreHeartbeat{} } func (m *LogStoreHeartbeat) String() string { return proto.CompactTextString(m) } func (*LogStoreHeartbeat) ProtoMessage() {} func (*LogStoreHeartbeat) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{8} + return fileDescriptor_fd1040c5381ab5a7, []int{9} } func (m *LogStoreHeartbeat) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -1604,6 +1784,13 @@ func (m *LogStoreHeartbeat) GetStoreIncarnation() string { return "" } +func (m *LogStoreHeartbeat) GetDDLVisibilityDeployedProtocolSupported() bool { + if m != nil { + return m.DDLVisibilityDeployedProtocolSupported + } + return false +} + // TNShardInfo contains information of a launched TN shard. type TNShardInfo struct { // ShardID uniquely identifies a TN shard. Each TN shard manages a Primary @@ -1622,7 +1809,7 @@ func (m *TNShardInfo) Reset() { *m = TNShardInfo{} } func (m *TNShardInfo) String() string { return proto.CompactTextString(m) } func (*TNShardInfo) ProtoMessage() {} func (*TNShardInfo) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{9} + return fileDescriptor_fd1040c5381ab5a7, []int{10} } func (m *TNShardInfo) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -1697,7 +1884,7 @@ func (m *TNStoreHeartbeat) Reset() { *m = TNStoreHeartbeat{} } func (m *TNStoreHeartbeat) String() string { return proto.CompactTextString(m) } func (*TNStoreHeartbeat) ProtoMessage() {} func (*TNStoreHeartbeat) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{10} + return fileDescriptor_fd1040c5381ab5a7, []int{11} } func (m *TNStoreHeartbeat) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -1842,7 +2029,7 @@ func (m *RSMState) Reset() { *m = RSMState{} } func (m *RSMState) String() string { return proto.CompactTextString(m) } func (*RSMState) ProtoMessage() {} func (*RSMState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{11} + return fileDescriptor_fd1040c5381ab5a7, []int{12} } func (m *RSMState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -1976,7 +2163,7 @@ func (m *LogRecord) Reset() { *m = LogRecord{} } func (m *LogRecord) String() string { return proto.CompactTextString(m) } func (*LogRecord) ProtoMessage() {} func (*LogRecord) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{12} + return fileDescriptor_fd1040c5381ab5a7, []int{13} } func (m *LogRecord) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2045,7 +2232,7 @@ func (m *LogRequest) Reset() { *m = LogRequest{} } func (m *LogRequest) String() string { return proto.CompactTextString(m) } func (*LogRequest) ProtoMessage() {} func (*LogRequest) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{13} + return fileDescriptor_fd1040c5381ab5a7, []int{14} } func (m *LogRequest) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2148,7 +2335,7 @@ func (m *TsoRequest) Reset() { *m = TsoRequest{} } func (m *TsoRequest) String() string { return proto.CompactTextString(m) } func (*TsoRequest) ProtoMessage() {} func (*TsoRequest) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{14} + return fileDescriptor_fd1040c5381ab5a7, []int{15} } func (m *TsoRequest) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2201,7 +2388,7 @@ func (m *CNStoreLabel) Reset() { *m = CNStoreLabel{} } func (m *CNStoreLabel) String() string { return proto.CompactTextString(m) } func (*CNStoreLabel) ProtoMessage() {} func (*CNStoreLabel) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{15} + return fileDescriptor_fd1040c5381ab5a7, []int{16} } func (m *CNStoreLabel) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2265,7 +2452,7 @@ func (m *CNWorkState) Reset() { *m = CNWorkState{} } func (m *CNWorkState) String() string { return proto.CompactTextString(m) } func (*CNWorkState) ProtoMessage() {} func (*CNWorkState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{16} + return fileDescriptor_fd1040c5381ab5a7, []int{17} } func (m *CNWorkState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2324,7 +2511,7 @@ func (m *CNStateLabel) Reset() { *m = CNStateLabel{} } func (m *CNStateLabel) String() string { return proto.CompactTextString(m) } func (*CNStateLabel) ProtoMessage() {} func (*CNStateLabel) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{17} + return fileDescriptor_fd1040c5381ab5a7, []int{18} } func (m *CNStateLabel) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2401,7 +2588,7 @@ func (m *Request) Reset() { *m = Request{} } func (m *Request) String() string { return proto.CompactTextString(m) } func (*Request) ProtoMessage() {} func (*Request) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{18} + return fileDescriptor_fd1040c5381ab5a7, []int{19} } func (m *Request) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2563,7 +2750,7 @@ func (m *LogResponse) Reset() { *m = LogResponse{} } func (m *LogResponse) String() string { return proto.CompactTextString(m) } func (*LogResponse) ProtoMessage() {} func (*LogResponse) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{19} + return fileDescriptor_fd1040c5381ab5a7, []int{20} } func (m *LogResponse) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2631,7 +2818,7 @@ func (m *AllocateIDResponse) Reset() { *m = AllocateIDResponse{} } func (m *AllocateIDResponse) String() string { return proto.CompactTextString(m) } func (*AllocateIDResponse) ProtoMessage() {} func (*AllocateIDResponse) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{20} + return fileDescriptor_fd1040c5381ab5a7, []int{21} } func (m *AllocateIDResponse) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2689,7 +2876,7 @@ func (m *Response) Reset() { *m = Response{} } func (m *Response) String() string { return proto.CompactTextString(m) } func (*Response) ProtoMessage() {} func (*Response) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{21} + return fileDescriptor_fd1040c5381ab5a7, []int{22} } func (m *Response) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2813,7 +3000,7 @@ func (m *LogRecordResponse) Reset() { *m = LogRecordResponse{} } func (m *LogRecordResponse) String() string { return proto.CompactTextString(m) } func (*LogRecordResponse) ProtoMessage() {} func (*LogRecordResponse) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{22} + return fileDescriptor_fd1040c5381ab5a7, []int{23} } func (m *LogRecordResponse) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2860,7 +3047,7 @@ func (m *TsoResponse) Reset() { *m = TsoResponse{} } func (m *TsoResponse) String() string { return proto.CompactTextString(m) } func (*TsoResponse) ProtoMessage() {} func (*TsoResponse) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{23} + return fileDescriptor_fd1040c5381ab5a7, []int{24} } func (m *TsoResponse) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2910,7 +3097,7 @@ func (m *TaskTableUser) Reset() { *m = TaskTableUser{} } func (m *TaskTableUser) String() string { return proto.CompactTextString(m) } func (*TaskTableUser) ProtoMessage() {} func (*TaskTableUser) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{24} + return fileDescriptor_fd1040c5381ab5a7, []int{25} } func (m *TaskTableUser) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -2971,7 +3158,7 @@ func (m *Replica) Reset() { *m = Replica{} } func (m *Replica) String() string { return proto.CompactTextString(m) } func (*Replica) ProtoMessage() {} func (*Replica) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{25} + return fileDescriptor_fd1040c5381ab5a7, []int{26} } func (m *Replica) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3050,7 +3237,7 @@ func (m *ConfigChange) Reset() { *m = ConfigChange{} } func (m *ConfigChange) String() string { return proto.CompactTextString(m) } func (*ConfigChange) ProtoMessage() {} func (*ConfigChange) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{26} + return fileDescriptor_fd1040c5381ab5a7, []int{27} } func (m *ConfigChange) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3112,7 +3299,7 @@ func (m *ShutdownStore) Reset() { *m = ShutdownStore{} } func (m *ShutdownStore) String() string { return proto.CompactTextString(m) } func (*ShutdownStore) ProtoMessage() {} func (*ShutdownStore) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{27} + return fileDescriptor_fd1040c5381ab5a7, []int{28} } func (m *ShutdownStore) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3171,7 +3358,7 @@ func (m *ScheduleCommand) Reset() { *m = ScheduleCommand{} } func (m *ScheduleCommand) String() string { return proto.CompactTextString(m) } func (*ScheduleCommand) ProtoMessage() {} func (*ScheduleCommand) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{28} + return fileDescriptor_fd1040c5381ab5a7, []int{29} } func (m *ScheduleCommand) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3291,7 +3478,7 @@ func (m *ScheduleCommandQuery) Reset() { *m = ScheduleCommandQuery{} } func (m *ScheduleCommandQuery) String() string { return proto.CompactTextString(m) } func (*ScheduleCommandQuery) ProtoMessage() {} func (*ScheduleCommandQuery) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{29} + return fileDescriptor_fd1040c5381ab5a7, []int{30} } func (m *ScheduleCommandQuery) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3359,7 +3546,7 @@ func (m *CommandDeliveryTargets) Reset() { *m = CommandDeliveryTargets{} func (m *CommandDeliveryTargets) String() string { return proto.CompactTextString(m) } func (*CommandDeliveryTargets) ProtoMessage() {} func (*CommandDeliveryTargets) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{30} + return fileDescriptor_fd1040c5381ab5a7, []int{31} } func (m *CommandDeliveryTargets) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3441,7 +3628,7 @@ func (m *JoinGossipCluster) Reset() { *m = JoinGossipCluster{} } func (m *JoinGossipCluster) String() string { return proto.CompactTextString(m) } func (*JoinGossipCluster) ProtoMessage() {} func (*JoinGossipCluster) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{31} + return fileDescriptor_fd1040c5381ab5a7, []int{32} } func (m *JoinGossipCluster) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3492,7 +3679,7 @@ func (m *CreateTaskService) Reset() { *m = CreateTaskService{} } func (m *CreateTaskService) String() string { return proto.CompactTextString(m) } func (*CreateTaskService) ProtoMessage() {} func (*CreateTaskService) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{32} + return fileDescriptor_fd1040c5381ab5a7, []int{33} } func (m *CreateTaskService) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3547,7 +3734,7 @@ func (m *DeleteCNStore) Reset() { *m = DeleteCNStore{} } func (m *DeleteCNStore) String() string { return proto.CompactTextString(m) } func (*DeleteCNStore) ProtoMessage() {} func (*DeleteCNStore) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{33} + return fileDescriptor_fd1040c5381ab5a7, []int{34} } func (m *DeleteCNStore) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3595,7 +3782,7 @@ func (m *DeleteProxyStore) Reset() { *m = DeleteProxyStore{} } func (m *DeleteProxyStore) String() string { return proto.CompactTextString(m) } func (*DeleteProxyStore) ProtoMessage() {} func (*DeleteProxyStore) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{34} + return fileDescriptor_fd1040c5381ab5a7, []int{35} } func (m *DeleteProxyStore) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3643,7 +3830,7 @@ func (m *AddLogShard) Reset() { *m = AddLogShard{} } func (m *AddLogShard) String() string { return proto.CompactTextString(m) } func (*AddLogShard) ProtoMessage() {} func (*AddLogShard) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{35} + return fileDescriptor_fd1040c5381ab5a7, []int{36} } func (m *AddLogShard) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3694,7 +3881,7 @@ func (m *BootstrapShard) Reset() { *m = BootstrapShard{} } func (m *BootstrapShard) String() string { return proto.CompactTextString(m) } func (*BootstrapShard) ProtoMessage() {} func (*BootstrapShard) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{36} + return fileDescriptor_fd1040c5381ab5a7, []int{37} } func (m *BootstrapShard) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3769,16 +3956,22 @@ type CommandBatch struct { // ViewMetadataAdmission is the authoritative snapshot for the heartbeat // sender. Old services ignore this additive field. ViewMetadataAdmission *ViewMetadataAdmission `protobuf:"bytes,5,opt,name=ViewMetadataAdmission,proto3" json:"ViewMetadataAdmission,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + // DDLVisibilityDeployedProtocol is returned from the same replicated + // heartbeat transition that conditionally publishes CN ingress. + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,6,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + // DDLVisibilityFrontier is the durable cluster maximum after applying this + // heartbeat transition. New CNs require it as publication acknowledgement. + DDLVisibilityFrontier *timestamp.Timestamp `protobuf:"bytes,7,opt,name=DDLVisibilityFrontier,proto3" json:"DDLVisibilityFrontier,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CommandBatch) Reset() { *m = CommandBatch{} } func (m *CommandBatch) String() string { return proto.CompactTextString(m) } func (*CommandBatch) ProtoMessage() {} func (*CommandBatch) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{37} + return fileDescriptor_fd1040c5381ab5a7, []int{38} } func (m *CommandBatch) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3842,6 +4035,20 @@ func (m *CommandBatch) GetViewMetadataAdmission() *ViewMetadataAdmission { return nil } +func (m *CommandBatch) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + +func (m *CommandBatch) GetDDLVisibilityFrontier() *timestamp.Timestamp { + if m != nil { + return m.DDLVisibilityFrontier + } + return nil +} + // ViewMetadataAdmission is the durable cluster admission snapshot. Generation // and Ready describe the heartbeat sender when returned in CommandBatch; they // are zero/false in the global ClusterDetails copy. @@ -3865,7 +4072,7 @@ func (m *ViewMetadataAdmission) Reset() { *m = ViewMetadataAdmission{} } func (m *ViewMetadataAdmission) String() string { return proto.CompactTextString(m) } func (*ViewMetadataAdmission) ProtoMessage() {} func (*ViewMetadataAdmission) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{38} + return fileDescriptor_fd1040c5381ab5a7, []int{39} } func (m *ViewMetadataAdmission) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -3966,7 +4173,7 @@ func (m *ViewMetadataAdmissionTargets) Reset() { *m = ViewMetadataAdmiss func (m *ViewMetadataAdmissionTargets) String() string { return proto.CompactTextString(m) } func (*ViewMetadataAdmissionTargets) ProtoMessage() {} func (*ViewMetadataAdmissionTargets) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{39} + return fileDescriptor_fd1040c5381ab5a7, []int{40} } func (m *ViewMetadataAdmissionTargets) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4044,25 +4251,30 @@ type CNStoreInfo struct { // CommandDeliveryAckSupported reports whether this CN can use the // non-destructive acknowledged schedule-command protocol. It is copied // from CNStoreHeartbeat so HAKeeper can gate activation and admission. - CommandDeliveryAckSupported bool `protobuf:"varint,18,opt,name=CommandDeliveryAckSupported,proto3" json:"CommandDeliveryAckSupported,omitempty"` - ViewMetadataAdmissionSupported bool `protobuf:"varint,19,opt,name=ViewMetadataAdmissionSupported,proto3" json:"ViewMetadataAdmissionSupported,omitempty"` - ViewMetadataAdmissionGeneration uint64 `protobuf:"varint,20,opt,name=ViewMetadataAdmissionGeneration,proto3" json:"ViewMetadataAdmissionGeneration,omitempty"` - ViewMetadataObservedEpoch uint64 `protobuf:"varint,21,opt,name=ViewMetadataObservedEpoch,proto3" json:"ViewMetadataObservedEpoch,omitempty"` - ViewMetadataCatalogFencedEpoch uint64 `protobuf:"varint,22,opt,name=ViewMetadataCatalogFencedEpoch,proto3" json:"ViewMetadataCatalogFencedEpoch,omitempty"` - ViewMetadataRefreshSupported bool `protobuf:"varint,23,opt,name=ViewMetadataRefreshSupported,proto3" json:"ViewMetadataRefreshSupported,omitempty"` - ViewMetadataRevalidatedEpoch uint64 `protobuf:"varint,24,opt,name=ViewMetadataRevalidatedEpoch,proto3" json:"ViewMetadataRevalidatedEpoch,omitempty"` - ViewMetadataAdmissionReady bool `protobuf:"varint,25,opt,name=ViewMetadataAdmissionReady,proto3" json:"ViewMetadataAdmissionReady,omitempty"` - ViewMetadataIngressReady bool `protobuf:"varint,26,opt,name=ViewMetadataIngressReady,proto3" json:"ViewMetadataIngressReady,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + CommandDeliveryAckSupported bool `protobuf:"varint,18,opt,name=CommandDeliveryAckSupported,proto3" json:"CommandDeliveryAckSupported,omitempty"` + ViewMetadataAdmissionSupported bool `protobuf:"varint,19,opt,name=ViewMetadataAdmissionSupported,proto3" json:"ViewMetadataAdmissionSupported,omitempty"` + ViewMetadataAdmissionGeneration uint64 `protobuf:"varint,20,opt,name=ViewMetadataAdmissionGeneration,proto3" json:"ViewMetadataAdmissionGeneration,omitempty"` + ViewMetadataObservedEpoch uint64 `protobuf:"varint,21,opt,name=ViewMetadataObservedEpoch,proto3" json:"ViewMetadataObservedEpoch,omitempty"` + ViewMetadataCatalogFencedEpoch uint64 `protobuf:"varint,22,opt,name=ViewMetadataCatalogFencedEpoch,proto3" json:"ViewMetadataCatalogFencedEpoch,omitempty"` + ViewMetadataRefreshSupported bool `protobuf:"varint,23,opt,name=ViewMetadataRefreshSupported,proto3" json:"ViewMetadataRefreshSupported,omitempty"` + ViewMetadataRevalidatedEpoch uint64 `protobuf:"varint,24,opt,name=ViewMetadataRevalidatedEpoch,proto3" json:"ViewMetadataRevalidatedEpoch,omitempty"` + ViewMetadataAdmissionReady bool `protobuf:"varint,25,opt,name=ViewMetadataAdmissionReady,proto3" json:"ViewMetadataAdmissionReady,omitempty"` + ViewMetadataIngressReady bool `protobuf:"varint,26,opt,name=ViewMetadataIngressReady,proto3" json:"ViewMetadataIngressReady,omitempty"` + DDLVisibilityBarrierReady bool `protobuf:"varint,27,opt,name=DDLVisibilityBarrierReady,proto3" json:"DDLVisibilityBarrierReady,omitempty"` + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,28,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + DDLVisibilityActivationPrepared bool `protobuf:"varint,29,opt,name=DDLVisibilityActivationPrepared,proto3" json:"DDLVisibilityActivationPrepared,omitempty"` + DDLVisibilityActivationFenced bool `protobuf:"varint,30,opt,name=DDLVisibilityActivationFenced,proto3" json:"DDLVisibilityActivationFenced,omitempty"` + DDLVisibilityFrontier timestamp.Timestamp `protobuf:"bytes,31,opt,name=DDLVisibilityFrontier,proto3" json:"DDLVisibilityFrontier"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CNStoreInfo) Reset() { *m = CNStoreInfo{} } func (m *CNStoreInfo) String() string { return proto.CompactTextString(m) } func (*CNStoreInfo) ProtoMessage() {} func (*CNStoreInfo) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{40} + return fileDescriptor_fd1040c5381ab5a7, []int{41} } func (m *CNStoreInfo) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4266,20 +4478,62 @@ func (m *CNStoreInfo) GetViewMetadataIngressReady() bool { return false } +func (m *CNStoreInfo) GetDDLVisibilityBarrierReady() bool { + if m != nil { + return m.DDLVisibilityBarrierReady + } + return false +} + +func (m *CNStoreInfo) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + +func (m *CNStoreInfo) GetDDLVisibilityActivationPrepared() bool { + if m != nil { + return m.DDLVisibilityActivationPrepared + } + return false +} + +func (m *CNStoreInfo) GetDDLVisibilityActivationFenced() bool { + if m != nil { + return m.DDLVisibilityActivationFenced + } + return false +} + +func (m *CNStoreInfo) GetDDLVisibilityFrontier() timestamp.Timestamp { + if m != nil { + return m.DDLVisibilityFrontier + } + return timestamp.Timestamp{} +} + // CNState contains all CN details known to the HAKeeper. type CNState struct { // Stores is keyed by CN store UUID. - Stores map[string]CNStoreInfo `protobuf:"bytes,1,rep,name=Stores,proto3" json:"Stores" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + Stores map[string]CNStoreInfo `protobuf:"bytes,1,rep,name=Stores,proto3" json:"Stores" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"` + // DDLVisibilityDeployedProtocol is a monotonic cluster deployment epoch. + // Once a complete-target cut commits, markerless replacement/scale-out CNs + // must join that fence instead of reopening on an older protocol. + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,2,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + // DDLVisibilityFrontier is the monotonic cluster-lifetime maximum committed + // DDL timestamp. It survives CN restart, replacement, and store removal. + DDLVisibilityFrontier timestamp.Timestamp `protobuf:"bytes,3,opt,name=DDLVisibilityFrontier,proto3" json:"DDLVisibilityFrontier"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CNState) Reset() { *m = CNState{} } func (m *CNState) String() string { return proto.CompactTextString(m) } func (*CNState) ProtoMessage() {} func (*CNState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{41} + return fileDescriptor_fd1040c5381ab5a7, []int{42} } func (m *CNState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4315,6 +4569,20 @@ func (m *CNState) GetStores() map[string]CNStoreInfo { return nil } +func (m *CNState) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + +func (m *CNState) GetDDLVisibilityFrontier() timestamp.Timestamp { + if m != nil { + return m.DDLVisibilityFrontier + } + return timestamp.Timestamp{} +} + // TNStoreInfo contains information on a TN store. type TNStoreInfo struct { Tick uint64 `protobuf:"varint,1,opt,name=Tick,proto3" json:"Tick,omitempty"` @@ -4344,7 +4612,7 @@ func (m *TNStoreInfo) Reset() { *m = TNStoreInfo{} } func (m *TNStoreInfo) String() string { return proto.CompactTextString(m) } func (*TNStoreInfo) ProtoMessage() {} func (*TNStoreInfo) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{42} + return fileDescriptor_fd1040c5381ab5a7, []int{43} } func (m *TNStoreInfo) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4470,7 +4738,7 @@ func (m *TNState) Reset() { *m = TNState{} } func (m *TNState) String() string { return proto.CompactTextString(m) } func (*TNState) ProtoMessage() {} func (*TNState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{43} + return fileDescriptor_fd1040c5381ab5a7, []int{44} } func (m *TNState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4524,7 +4792,7 @@ func (m *ProxyStore) Reset() { *m = ProxyStore{} } func (m *ProxyStore) String() string { return proto.CompactTextString(m) } func (*ProxyStore) ProtoMessage() {} func (*ProxyStore) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{44} + return fileDescriptor_fd1040c5381ab5a7, []int{45} } func (m *ProxyStore) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4620,7 +4888,7 @@ func (m *ProxyState) Reset() { *m = ProxyState{} } func (m *ProxyState) String() string { return proto.CompactTextString(m) } func (*ProxyState) ProtoMessage() {} func (*ProxyState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{45} + return fileDescriptor_fd1040c5381ab5a7, []int{46} } func (m *ProxyState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4672,7 +4940,7 @@ func (m *ProxyHeartbeat) Reset() { *m = ProxyHeartbeat{} } func (m *ProxyHeartbeat) String() string { return proto.CompactTextString(m) } func (*ProxyHeartbeat) ProtoMessage() {} func (*ProxyHeartbeat) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{46} + return fileDescriptor_fd1040c5381ab5a7, []int{47} } func (m *ProxyHeartbeat) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4744,22 +5012,24 @@ func (m *ProxyHeartbeat) GetViewMetadataObservedEpoch() uint64 { } type ClusterDetails struct { - TNStores []TNStore `protobuf:"bytes,1,rep,name=TNStores,proto3" json:"TNStores"` - CNStores []CNStore `protobuf:"bytes,2,rep,name=CNStores,proto3" json:"CNStores"` - LogStores []LogStore `protobuf:"bytes,3,rep,name=LogStores,proto3" json:"LogStores"` - ProxyStores []ProxyStore `protobuf:"bytes,4,rep,name=ProxyStores,proto3" json:"ProxyStores"` - DeletedStores []DeletedStore `protobuf:"bytes,5,rep,name=DeletedStores,proto3" json:"DeletedStores"` - ViewMetadataAdmission *ViewMetadataAdmission `protobuf:"bytes,6,opt,name=ViewMetadataAdmission,proto3" json:"ViewMetadataAdmission,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + TNStores []TNStore `protobuf:"bytes,1,rep,name=TNStores,proto3" json:"TNStores"` + CNStores []CNStore `protobuf:"bytes,2,rep,name=CNStores,proto3" json:"CNStores"` + LogStores []LogStore `protobuf:"bytes,3,rep,name=LogStores,proto3" json:"LogStores"` + ProxyStores []ProxyStore `protobuf:"bytes,4,rep,name=ProxyStores,proto3" json:"ProxyStores"` + DeletedStores []DeletedStore `protobuf:"bytes,5,rep,name=DeletedStores,proto3" json:"DeletedStores"` + ViewMetadataAdmission *ViewMetadataAdmission `protobuf:"bytes,6,opt,name=ViewMetadataAdmission,proto3" json:"ViewMetadataAdmission,omitempty"` + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,7,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + DDLVisibilityFrontier timestamp.Timestamp `protobuf:"bytes,8,opt,name=DDLVisibilityFrontier,proto3" json:"DDLVisibilityFrontier"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *ClusterDetails) Reset() { *m = ClusterDetails{} } func (m *ClusterDetails) String() string { return proto.CompactTextString(m) } func (*ClusterDetails) ProtoMessage() {} func (*ClusterDetails) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{47} + return fileDescriptor_fd1040c5381ab5a7, []int{48} } func (m *ClusterDetails) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4830,6 +5100,20 @@ func (m *ClusterDetails) GetViewMetadataAdmission() *ViewMetadataAdmission { return nil } +func (m *ClusterDetails) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + +func (m *ClusterDetails) GetDDLVisibilityFrontier() timestamp.Timestamp { + if m != nil { + return m.DDLVisibilityFrontier + } + return timestamp.Timestamp{} +} + // ClusterInfo provides a global view of all shards in the cluster. It // describes the logical sharding of the system, rather than physical // distribution of all replicas that belong to those shards. @@ -4845,7 +5129,7 @@ func (m *ClusterInfo) Reset() { *m = ClusterInfo{} } func (m *ClusterInfo) String() string { return proto.CompactTextString(m) } func (*ClusterInfo) ProtoMessage() {} func (*ClusterInfo) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{48} + return fileDescriptor_fd1040c5381ab5a7, []int{49} } func (m *ClusterInfo) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4905,7 +5189,7 @@ func (m *InitialClusterRequest) Reset() { *m = InitialClusterRequest{} } func (m *InitialClusterRequest) String() string { return proto.CompactTextString(m) } func (*InitialClusterRequest) ProtoMessage() {} func (*InitialClusterRequest) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{49} + return fileDescriptor_fd1040c5381ab5a7, []int{50} } func (m *InitialClusterRequest) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -4996,7 +5280,7 @@ func (m *RestoreIDWatermarkRequest) Reset() { *m = RestoreIDWatermarkReq func (m *RestoreIDWatermarkRequest) String() string { return proto.CompactTextString(m) } func (*RestoreIDWatermarkRequest) ProtoMessage() {} func (*RestoreIDWatermarkRequest) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{50} + return fileDescriptor_fd1040c5381ab5a7, []int{51} } func (m *RestoreIDWatermarkRequest) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5048,27 +5332,28 @@ func (m *RestoreIDWatermarkRequest) GetLogServiceRecovery() bool { // LogStoreInfo contains information of all replicas found on a Log store. type LogStoreInfo struct { - Tick uint64 `protobuf:"varint,1,opt,name=Tick,proto3" json:"Tick,omitempty"` - RaftAddress string `protobuf:"bytes,2,opt,name=RaftAddress,proto3" json:"RaftAddress,omitempty"` - ServiceAddress string `protobuf:"bytes,3,opt,name=ServiceAddress,proto3" json:"ServiceAddress,omitempty"` - GossipAddress string `protobuf:"bytes,4,opt,name=GossipAddress,proto3" json:"GossipAddress,omitempty"` - Replicas []LogReplicaInfo `protobuf:"bytes,5,rep,name=Replicas,proto3" json:"Replicas"` - TaskServiceCreated bool `protobuf:"varint,6,opt,name=TaskServiceCreated,proto3" json:"TaskServiceCreated,omitempty"` - ConfigData *ConfigData `protobuf:"bytes,7,opt,name=ConfigData,proto3" json:"ConfigData,omitempty"` - Locality Locality `protobuf:"bytes,8,opt,name=Locality,proto3" json:"Locality"` - CommandDeliverySupported bool `protobuf:"varint,9,opt,name=CommandDeliverySupported,proto3" json:"CommandDeliverySupported,omitempty"` - ViewMetadataAdmissionSupported bool `protobuf:"varint,10,opt,name=ViewMetadataAdmissionSupported,proto3" json:"ViewMetadataAdmissionSupported,omitempty"` - StoreIncarnation string `protobuf:"bytes,11,opt,name=StoreIncarnation,proto3" json:"StoreIncarnation,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + Tick uint64 `protobuf:"varint,1,opt,name=Tick,proto3" json:"Tick,omitempty"` + RaftAddress string `protobuf:"bytes,2,opt,name=RaftAddress,proto3" json:"RaftAddress,omitempty"` + ServiceAddress string `protobuf:"bytes,3,opt,name=ServiceAddress,proto3" json:"ServiceAddress,omitempty"` + GossipAddress string `protobuf:"bytes,4,opt,name=GossipAddress,proto3" json:"GossipAddress,omitempty"` + Replicas []LogReplicaInfo `protobuf:"bytes,5,rep,name=Replicas,proto3" json:"Replicas"` + TaskServiceCreated bool `protobuf:"varint,6,opt,name=TaskServiceCreated,proto3" json:"TaskServiceCreated,omitempty"` + ConfigData *ConfigData `protobuf:"bytes,7,opt,name=ConfigData,proto3" json:"ConfigData,omitempty"` + Locality Locality `protobuf:"bytes,8,opt,name=Locality,proto3" json:"Locality"` + CommandDeliverySupported bool `protobuf:"varint,9,opt,name=CommandDeliverySupported,proto3" json:"CommandDeliverySupported,omitempty"` + ViewMetadataAdmissionSupported bool `protobuf:"varint,10,opt,name=ViewMetadataAdmissionSupported,proto3" json:"ViewMetadataAdmissionSupported,omitempty"` + StoreIncarnation string `protobuf:"bytes,11,opt,name=StoreIncarnation,proto3" json:"StoreIncarnation,omitempty"` + DDLVisibilityDeployedProtocolSupported bool `protobuf:"varint,12,opt,name=DDLVisibilityDeployedProtocolSupported,proto3" json:"DDLVisibilityDeployedProtocolSupported,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *LogStoreInfo) Reset() { *m = LogStoreInfo{} } func (m *LogStoreInfo) String() string { return proto.CompactTextString(m) } func (*LogStoreInfo) ProtoMessage() {} func (*LogStoreInfo) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{51} + return fileDescriptor_fd1040c5381ab5a7, []int{52} } func (m *LogStoreInfo) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5174,6 +5459,13 @@ func (m *LogStoreInfo) GetStoreIncarnation() string { return "" } +func (m *LogStoreInfo) GetDDLVisibilityDeployedProtocolSupported() bool { + if m != nil { + return m.DDLVisibilityDeployedProtocolSupported + } + return false +} + type LogState struct { // Shards is keyed by ShardID, it contains details aggregated from all Log // stores. Each pb.LogShardInfo here contains data aggregated from @@ -5193,7 +5485,7 @@ func (m *LogState) Reset() { *m = LogState{} } func (m *LogState) String() string { return proto.CompactTextString(m) } func (*LogState) ProtoMessage() {} func (*LogState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{52} + return fileDescriptor_fd1040c5381ab5a7, []int{53} } func (m *LogState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5266,7 +5558,7 @@ func (m *CheckerState) Reset() { *m = CheckerState{} } func (m *CheckerState) String() string { return proto.CompactTextString(m) } func (*CheckerState) ProtoMessage() {} func (*CheckerState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{53} + return fileDescriptor_fd1040c5381ab5a7, []int{54} } func (m *CheckerState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5437,7 +5729,7 @@ func (m *DeletedStore) Reset() { *m = DeletedStore{} } func (m *DeletedStore) String() string { return proto.CompactTextString(m) } func (*DeletedStore) ProtoMessage() {} func (*DeletedStore) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{54} + return fileDescriptor_fd1040c5381ab5a7, []int{55} } func (m *DeletedStore) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5584,7 +5876,7 @@ func (m *HAKeeperRSMState) Reset() { *m = HAKeeperRSMState{} } func (m *HAKeeperRSMState) String() string { return proto.CompactTextString(m) } func (*HAKeeperRSMState) ProtoMessage() {} func (*HAKeeperRSMState) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{55} + return fileDescriptor_fd1040c5381ab5a7, []int{56} } func (m *HAKeeperRSMState) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5920,7 +6212,7 @@ func (m *ReplicaInfo) Reset() { *m = ReplicaInfo{} } func (m *ReplicaInfo) String() string { return proto.CompactTextString(m) } func (*ReplicaInfo) ProtoMessage() {} func (*ReplicaInfo) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{56} + return fileDescriptor_fd1040c5381ab5a7, []int{57} } func (m *ReplicaInfo) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -5979,7 +6271,7 @@ func (m *ShardInfoQueryResult) Reset() { *m = ShardInfoQueryResult{} } func (m *ShardInfoQueryResult) String() string { return proto.CompactTextString(m) } func (*ShardInfoQueryResult) ProtoMessage() {} func (*ShardInfoQueryResult) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{57} + return fileDescriptor_fd1040c5381ab5a7, []int{58} } func (m *ShardInfoQueryResult) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6059,7 +6351,7 @@ func (m *BackupData) Reset() { *m = BackupData{} } func (m *BackupData) String() string { return proto.CompactTextString(m) } func (*BackupData) ProtoMessage() {} func (*BackupData) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{58} + return fileDescriptor_fd1040c5381ab5a7, []int{59} } func (m *BackupData) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6120,7 +6412,7 @@ func (m *ConfigItem) Reset() { *m = ConfigItem{} } func (m *ConfigItem) String() string { return proto.CompactTextString(m) } func (*ConfigItem) ProtoMessage() {} func (*ConfigItem) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{59} + return fileDescriptor_fd1040c5381ab5a7, []int{60} } func (m *ConfigItem) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6190,7 +6482,7 @@ func (m *ConfigData) Reset() { *m = ConfigData{} } func (m *ConfigData) String() string { return proto.CompactTextString(m) } func (*ConfigData) ProtoMessage() {} func (*ConfigData) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{60} + return fileDescriptor_fd1040c5381ab5a7, []int{61} } func (m *ConfigData) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6239,7 +6531,7 @@ func (m *Locality) Reset() { *m = Locality{} } func (m *Locality) String() string { return proto.CompactTextString(m) } func (*Locality) ProtoMessage() {} func (*Locality) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{61} + return fileDescriptor_fd1040c5381ab5a7, []int{62} } func (m *Locality) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6286,7 +6578,7 @@ func (m *CheckHealth) Reset() { *m = CheckHealth{} } func (m *CheckHealth) String() string { return proto.CompactTextString(m) } func (*CheckHealth) ProtoMessage() {} func (*CheckHealth) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{62} + return fileDescriptor_fd1040c5381ab5a7, []int{63} } func (m *CheckHealth) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6338,7 +6630,7 @@ func (m *ScheduleCommandID) Reset() { *m = ScheduleCommandID{} } func (m *ScheduleCommandID) String() string { return proto.CompactTextString(m) } func (*ScheduleCommandID) ProtoMessage() {} func (*ScheduleCommandID) Descriptor() ([]byte, []int) { - return fileDescriptor_fd1040c5381ab5a7, []int{63} + return fileDescriptor_fd1040c5381ab5a7, []int{64} } func (m *ScheduleCommandID) XXX_Unmarshal(b []byte) error { return m.Unmarshal(b) @@ -6402,6 +6694,7 @@ func init() { proto.RegisterMapType((map[uint64]string)(nil), "logservice.LogShardInfo.ReplicasEntry") proto.RegisterType((*LogReplicaInfo)(nil), "logservice.LogReplicaInfo") proto.RegisterType((*Resource)(nil), "logservice.Resource") + proto.RegisterType((*DDLVisibilityActivationTarget)(nil), "logservice.DDLVisibilityActivationTarget") proto.RegisterType((*CNStoreHeartbeat)(nil), "logservice.CNStoreHeartbeat") proto.RegisterType((*CNAllocateID)(nil), "logservice.CNAllocateID") proto.RegisterType((*LogStoreHeartbeat)(nil), "logservice.LogStoreHeartbeat") @@ -6497,381 +6790,399 @@ func init() { func init() { proto.RegisterFile("logservice.proto", fileDescriptor_fd1040c5381ab5a7) } var fileDescriptor_fd1040c5381ab5a7 = []byte{ - // 5969 bytes of a gzipped FileDescriptorProto - 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xec, 0x3d, 0x4d, 0x6f, 0x23, 0xc9, - 0x75, 0xd3, 0x14, 0x25, 0x91, 0x8f, 0x92, 0xa6, 0x55, 0xfa, 0x18, 0x8e, 0x66, 0x46, 0xa3, 0xe5, - 0x8e, 0x77, 0x67, 0xe5, 0xb5, 0xc6, 0x99, 0xf1, 0x2e, 0xd6, 0xce, 0x78, 0x76, 0x29, 0x92, 0x3b, - 0xe2, 0x0c, 0x45, 0x69, 0x9b, 0xad, 0x5d, 0xdb, 0xc0, 0x42, 0x68, 0x91, 0x35, 0x1a, 0x5a, 0x14, - 0x9b, 0xee, 0x6e, 0xcd, 0x47, 0x72, 0x0b, 0xe2, 0x20, 0x88, 0x81, 0x20, 0x06, 0x82, 0xd8, 0x49, - 0xec, 0xc4, 0x09, 0x90, 0x43, 0x4e, 0xbe, 0x24, 0x7f, 0x21, 0xf0, 0xc9, 0x30, 0x0c, 0xe4, 0x12, - 0x20, 0x8e, 0xed, 0x5c, 0x02, 0xf8, 0x1e, 0x9f, 0x02, 0x04, 0xf5, 0xd5, 0x5d, 0xd5, 0x5d, 0x4d, - 0x52, 0x1f, 0xeb, 0xc4, 0x89, 0x4f, 0x62, 0xbd, 0x8f, 0xfa, 0x7c, 0xef, 0xd5, 0xab, 0x57, 0xaf, - 0x5a, 0x60, 0xf6, 0xdc, 0x43, 0x1f, 0x7b, 0xcf, 0xba, 0x6d, 0xbc, 0x31, 0xf0, 0xdc, 0xc0, 0x45, - 0x10, 0x41, 0x56, 0x3e, 0x73, 0xd8, 0x0d, 0x9e, 0x9e, 0x1c, 0x6c, 0xb4, 0xdd, 0xe3, 0x3b, 0x87, - 0xee, 0xa1, 0x7b, 0x87, 0x92, 0x1c, 0x9c, 0x3c, 0xa1, 0x25, 0x5a, 0xa0, 0xbf, 0x18, 0xeb, 0xca, - 0xcd, 0x43, 0xd7, 0x3d, 0xec, 0xe1, 0x88, 0x2a, 0xe8, 0x1e, 0x63, 0x3f, 0x70, 0x8e, 0x07, 0x9c, - 0x60, 0xee, 0x18, 0x07, 0x4e, 0xc7, 0x09, 0x1c, 0x56, 0x2e, 0xfd, 0x70, 0x1a, 0xa6, 0x2b, 0xcd, - 0x56, 0xe0, 0x7a, 0x18, 0x21, 0xc8, 0xee, 0xed, 0xd5, 0xab, 0x45, 0x63, 0xcd, 0xb8, 0x9d, 0xb7, - 0xe8, 0x6f, 0xf4, 0x1a, 0xcc, 0xb5, 0x58, 0x57, 0xca, 0x9d, 0x8e, 0x87, 0x7d, 0xbf, 0x98, 0xa1, - 0xd8, 0x18, 0x14, 0xad, 0x02, 0xb4, 0x3e, 0x68, 0x08, 0x9a, 0x09, 0x4a, 0x23, 0x41, 0xd0, 0x06, - 0xa0, 0x86, 0xdb, 0x3e, 0x8a, 0xd5, 0x95, 0xa5, 0x74, 0x1a, 0x0c, 0xba, 0x05, 0x59, 0xcb, 0xed, - 0xe1, 0xe2, 0xd4, 0x9a, 0x71, 0x7b, 0xee, 0xae, 0xb9, 0x11, 0x76, 0xbb, 0xd2, 0x24, 0x70, 0x8b, - 0x62, 0x49, 0x8f, 0xed, 0x6e, 0xfb, 0xa8, 0x38, 0xbd, 0x66, 0xdc, 0xce, 0x5a, 0xf4, 0x37, 0xfa, - 0x34, 0x4c, 0xb6, 0x02, 0x27, 0xc0, 0xc5, 0x1c, 0x65, 0x5d, 0xda, 0x90, 0xe6, 0xb7, 0xe9, 0x76, - 0x30, 0x45, 0x5a, 0x8c, 0x06, 0x7d, 0x11, 0xa6, 0x1a, 0xce, 0x01, 0xee, 0xf9, 0xc5, 0xfc, 0xda, - 0xc4, 0xed, 0xc2, 0xdd, 0x9b, 0x32, 0x35, 0x9f, 0x97, 0x0d, 0x46, 0x51, 0xeb, 0x07, 0xde, 0xcb, - 0xcd, 0xec, 0x0f, 0x7e, 0x72, 0xf3, 0x92, 0xc5, 0x99, 0xd0, 0x6f, 0x41, 0xfe, 0x23, 0xd7, 0x3b, - 0x62, 0xed, 0x01, 0x6d, 0x6f, 0x21, 0xea, 0x6a, 0x88, 0xb2, 0x22, 0x2a, 0x54, 0x82, 0x99, 0x0f, - 0x4e, 0xb0, 0xf7, 0x52, 0x4c, 0x41, 0x81, 0x4e, 0x81, 0x02, 0x43, 0x6f, 0x03, 0x54, 0xdc, 0xfe, - 0x93, 0xee, 0x61, 0xd5, 0x09, 0x9c, 0xe2, 0xcc, 0x9a, 0x71, 0xbb, 0x70, 0x77, 0x59, 0xe9, 0x59, - 0x88, 0xb5, 0x24, 0x4a, 0xf4, 0x36, 0xe4, 0x2c, 0xec, 0xbb, 0x27, 0x5e, 0x1b, 0x17, 0x67, 0x29, - 0xd7, 0xa2, 0xcc, 0x25, 0x70, 0x7c, 0x10, 0x21, 0x2d, 0x5a, 0x86, 0xa9, 0xbd, 0x81, 0xdd, 0x3d, - 0xc6, 0xc5, 0xb9, 0x35, 0xe3, 0xf6, 0x84, 0xc5, 0x4b, 0xe8, 0xb3, 0xb0, 0xd0, 0x7a, 0xea, 0x78, - 0x9d, 0xd8, 0xaa, 0x5d, 0xa6, 0x5d, 0xd6, 0xa1, 0xd0, 0x0a, 0xe4, 0x2a, 0xee, 0xf1, 0x71, 0x37, - 0xa8, 0x57, 0x8b, 0x26, 0x25, 0x0b, 0xcb, 0xe8, 0x7d, 0x58, 0xfd, 0xb0, 0x8b, 0x9f, 0x6f, 0xf3, - 0xe9, 0x29, 0x77, 0x8e, 0xbb, 0xbe, 0xdf, 0x75, 0xfb, 0xad, 0x93, 0xc1, 0xc0, 0xf5, 0x02, 0xdc, - 0x29, 0xce, 0xaf, 0x19, 0xb7, 0x73, 0xd6, 0x08, 0x2a, 0xb4, 0x05, 0x37, 0xb5, 0x14, 0x0f, 0x71, - 0x1f, 0x7b, 0x4e, 0xd0, 0x75, 0xfb, 0x45, 0x44, 0xe5, 0x61, 0x14, 0x19, 0xba, 0x0f, 0x57, 0x65, - 0x92, 0x9d, 0x03, 0x32, 0x53, 0xb8, 0x53, 0x1b, 0xb8, 0xed, 0xa7, 0xc5, 0x05, 0x5a, 0x47, 0x3a, - 0x01, 0x7a, 0x00, 0x2b, 0xda, 0x06, 0x2c, 0xec, 0x74, 0x5e, 0x16, 0x17, 0xe9, 0x58, 0x86, 0x50, - 0xac, 0x34, 0xa1, 0x20, 0x49, 0x16, 0x32, 0x61, 0xe2, 0x08, 0xbf, 0xe4, 0xca, 0x47, 0x7e, 0xa2, - 0x37, 0x60, 0xf2, 0x99, 0xd3, 0x3b, 0xc1, 0x54, 0xe5, 0x0a, 0xb2, 0x64, 0x51, 0xbe, 0x46, 0xd7, - 0x0f, 0x2c, 0x46, 0xf1, 0x85, 0xcc, 0x3b, 0xc6, 0xa3, 0x6c, 0x6e, 0xd2, 0x9c, 0x2a, 0xfd, 0x72, - 0x02, 0xa6, 0xed, 0x0b, 0x50, 0x68, 0xa1, 0x5a, 0x13, 0x3a, 0xd5, 0xca, 0x8e, 0xa1, 0x5a, 0x6f, - 0xc1, 0x14, 0x95, 0x10, 0xbf, 0x38, 0x49, 0x55, 0xeb, 0x8a, 0x4c, 0x6d, 0x37, 0x29, 0xae, 0xde, - 0x7f, 0xe2, 0x0a, 0x95, 0x62, 0xc4, 0xe8, 0x2e, 0x2c, 0x36, 0xdc, 0xc3, 0xc0, 0xe9, 0xf6, 0x48, - 0x87, 0xb0, 0x27, 0x7a, 0x39, 0x45, 0x7b, 0xa9, 0xc5, 0xa5, 0x18, 0x97, 0xe9, 0x54, 0xe3, 0xa2, - 0xea, 0x57, 0x7e, 0x6c, 0xfd, 0x8a, 0xeb, 0x2e, 0x68, 0x74, 0x37, 0x45, 0x67, 0x0a, 0xe9, 0x3a, - 0xf3, 0x1e, 0x5c, 0x2b, 0x9f, 0x04, 0x6e, 0xbd, 0xdf, 0xf6, 0xa8, 0x60, 0xbd, 0x8f, 0xfb, 0x6d, - 0x1c, 0x29, 0xc5, 0x0c, 0x15, 0xa4, 0x61, 0x24, 0x8f, 0xb2, 0xb9, 0x9c, 0x99, 0x2f, 0xfd, 0x63, - 0x06, 0x72, 0x0d, 0xf7, 0xf0, 0x7f, 0xc1, 0xd2, 0xdf, 0x27, 0x76, 0x68, 0xd0, 0xeb, 0xb6, 0x1d, - 0xb1, 0xf8, 0x2b, 0x32, 0x7d, 0xc3, 0x3d, 0xe4, 0x68, 0x69, 0xfd, 0x43, 0x8e, 0xd8, 0xea, 0x4c, - 0x9d, 0xc6, 0xfa, 0x35, 0xdc, 0xb6, 0xd3, 0xeb, 0x06, 0x2f, 0xe9, 0xda, 0xc7, 0xac, 0x9f, 0xc0, - 0x89, 0xf6, 0x44, 0xb9, 0xf4, 0xcf, 0x59, 0x98, 0x21, 0xf3, 0x26, 0x04, 0x12, 0x15, 0x61, 0x9a, - 0x15, 0xd8, 0xf4, 0x65, 0x2d, 0x51, 0x44, 0x9b, 0xd2, 0xc0, 0x32, 0x74, 0x60, 0xaf, 0xc5, 0x06, - 0x16, 0xd6, 0xb2, 0x21, 0x08, 0xa9, 0x76, 0x4b, 0xc3, 0x5b, 0x84, 0x49, 0x66, 0x60, 0xd8, 0xf4, - 0xb2, 0x02, 0x31, 0x9c, 0x0d, 0xec, 0x74, 0xb0, 0x57, 0xaf, 0xd2, 0x29, 0xce, 0x5a, 0x61, 0x99, - 0xae, 0x07, 0xf6, 0x8e, 0x8b, 0x93, 0x7c, 0x3d, 0xb0, 0x77, 0x8c, 0x3e, 0x86, 0xf9, 0xa6, 0xdb, - 0xff, 0xd0, 0x0d, 0xba, 0xfd, 0xc3, 0xb0, 0x4b, 0x53, 0xb4, 0x4b, 0x77, 0x52, 0xbb, 0x94, 0xe0, - 0x60, 0x7d, 0x4b, 0xd6, 0x84, 0x3c, 0x28, 0xf2, 0xdf, 0x54, 0x9c, 0xea, 0xfd, 0xb6, 0xe3, 0xf5, - 0xa9, 0xd1, 0x24, 0x7a, 0x45, 0x5a, 0x79, 0x7b, 0xd4, 0xc0, 0x13, 0x8c, 0xac, 0xb1, 0xd4, 0x7a, - 0x57, 0x7e, 0x1b, 0x66, 0x95, 0x7e, 0xc9, 0x16, 0x31, 0xcb, 0x2c, 0xe2, 0xa2, 0x6c, 0x11, 0xf3, - 0x92, 0xf1, 0x5b, 0xa9, 0xc2, 0xb2, 0x7e, 0x74, 0xa7, 0xaa, 0xe5, 0x31, 0xdc, 0x18, 0xda, 0xfb, - 0xd3, 0x54, 0x56, 0xfa, 0xb6, 0x01, 0x73, 0xaa, 0xa8, 0xa3, 0xf7, 0x55, 0x49, 0xa3, 0xf5, 0x14, - 0xee, 0x16, 0xd3, 0xa6, 0x72, 0x33, 0x47, 0x44, 0xf5, 0x47, 0x3f, 0xb9, 0x69, 0x58, 0xaa, 0x84, - 0x5e, 0x87, 0xbc, 0xa8, 0xb6, 0x4a, 0x1b, 0xce, 0x5a, 0x11, 0x00, 0xad, 0x41, 0xa1, 0xee, 0x87, - 0xb3, 0x41, 0xe5, 0x2c, 0x67, 0xc9, 0xa0, 0xd2, 0x1f, 0x19, 0x91, 0xa7, 0x40, 0xf7, 0xec, 0xdd, - 0x3d, 0xdb, 0x0d, 0x9c, 0x1e, 0x1f, 0x58, 0x58, 0x26, 0x16, 0xaf, 0xb2, 0xbb, 0x57, 0x7e, 0xe6, - 0x74, 0x7b, 0xce, 0x41, 0x8f, 0x0d, 0xd2, 0xb0, 0x14, 0x18, 0xe1, 0xdf, 0xc6, 0xc7, 0x8c, 0x9f, - 0xc9, 0x74, 0x58, 0x26, 0xfc, 0xdb, 0xf8, 0x38, 0xe2, 0x67, 0xa2, 0xad, 0xc0, 0x4a, 0xdf, 0xc9, - 0x83, 0xc9, 0x5d, 0xad, 0x2d, 0xec, 0x78, 0xc1, 0x01, 0x76, 0x82, 0x5f, 0x43, 0x5f, 0x74, 0x03, - 0x90, 0xed, 0xf8, 0x82, 0xb7, 0xe2, 0x61, 0x87, 0x58, 0xef, 0x69, 0x3a, 0xf9, 0x1a, 0x4c, 0x62, - 0x33, 0xc9, 0x69, 0x36, 0x93, 0x5b, 0x30, 0x5b, 0xef, 0x77, 0x83, 0xc8, 0xc7, 0xcc, 0x53, 0x22, - 0x15, 0x48, 0xa8, 0x1e, 0xba, 0xbe, 0xdf, 0x1d, 0xa8, 0xfb, 0x92, 0x0a, 0x24, 0xed, 0x31, 0xc0, - 0x23, 0xb7, 0xdb, 0xc7, 0x1d, 0xba, 0x23, 0xe5, 0x2c, 0x05, 0xf6, 0x2b, 0x77, 0x3c, 0x53, 0x36, - 0xcb, 0xb9, 0xf1, 0x1c, 0xcc, 0xcb, 0x31, 0x07, 0xf3, 0xb3, 0xb0, 0x50, 0x6e, 0x1f, 0xe1, 0x0e, - 0x01, 0x38, 0xfd, 0xce, 0xa6, 0x13, 0xb4, 0x9f, 0x72, 0x3f, 0x34, 0x6b, 0xe9, 0x50, 0x64, 0xeb, - 0xe5, 0x90, 0x2a, 0xee, 0x75, 0x9f, 0x91, 0x99, 0x6f, 0x1f, 0xc5, 0xfd, 0xd1, 0x61, 0x24, 0x63, - 0x38, 0xb5, 0xe8, 0xa2, 0x9c, 0xda, 0x85, 0x0b, 0x70, 0x6a, 0x17, 0x47, 0x39, 0xb5, 0xb1, 0xf1, - 0x54, 0x9c, 0xc0, 0xe9, 0xb9, 0x87, 0xd4, 0xdf, 0xe0, 0x55, 0x2c, 0xd1, 0x2a, 0x46, 0x50, 0xa1, - 0x4d, 0xb8, 0x2e, 0x53, 0x58, 0xf8, 0x89, 0x87, 0xfd, 0xa7, 0xd1, 0xac, 0x2c, 0xd3, 0x59, 0x19, - 0x4a, 0x93, 0xac, 0xe3, 0x99, 0xd3, 0xeb, 0x76, 0x88, 0xf2, 0xb0, 0x9e, 0x5c, 0xa1, 0x3d, 0x19, - 0x4a, 0x83, 0xbe, 0x00, 0x45, 0x19, 0x5f, 0xef, 0x1f, 0x12, 0x31, 0x62, 0x2e, 0x7a, 0x91, 0xf6, - 0x21, 0x15, 0xcf, 0x1d, 0x6a, 0x1b, 0x66, 0x2a, 0xcd, 0x72, 0xaf, 0xe7, 0xb6, 0x9d, 0x00, 0xd7, - 0xab, 0x1a, 0x3f, 0x7d, 0x11, 0x26, 0xa9, 0x40, 0x71, 0x4b, 0xcc, 0x0a, 0xcc, 0x46, 0x7f, 0xed, - 0x04, 0xfb, 0x44, 0x54, 0x99, 0x11, 0x8a, 0x00, 0xa5, 0xbf, 0xcb, 0xc2, 0xbc, 0x70, 0xd6, 0x86, - 0x5b, 0xbd, 0x35, 0x28, 0x58, 0xce, 0x93, 0x40, 0x35, 0x79, 0x32, 0x48, 0x63, 0x17, 0x27, 0xb4, - 0x76, 0x31, 0x61, 0x27, 0xb2, 0x3a, 0x3b, 0x71, 0x3e, 0xe7, 0x4d, 0x6f, 0x05, 0xa7, 0x52, 0xad, - 0xa0, 0x6a, 0x71, 0xa6, 0xcf, 0xe4, 0xec, 0xe5, 0xc6, 0x77, 0xf6, 0x88, 0x3c, 0xc4, 0xd4, 0x39, - 0x92, 0xc9, 0x3c, 0x93, 0x87, 0x34, 0xfc, 0x18, 0xba, 0x0e, 0x63, 0xe9, 0xfa, 0x3a, 0x98, 0x71, - 0xf7, 0x82, 0x9f, 0x0f, 0x12, 0xf0, 0x52, 0x0d, 0x0a, 0xd2, 0x59, 0x69, 0x88, 0x6b, 0x3a, 0xd4, - 0x25, 0x28, 0x7d, 0x7d, 0x12, 0x4c, 0xfb, 0x22, 0xf7, 0xd8, 0xe8, 0x74, 0x37, 0x71, 0x9a, 0xd3, - 0x9d, 0x5e, 0x3c, 0xb2, 0xa9, 0xe2, 0x91, 0x76, 0x1a, 0x9c, 0x3c, 0xf5, 0x69, 0x70, 0x6a, 0xcc, - 0xd3, 0x60, 0xee, 0xcc, 0xa7, 0xc1, 0xfc, 0xf8, 0xa7, 0x41, 0x48, 0xdf, 0xe0, 0x88, 0xba, 0xe3, - 0x41, 0xcf, 0x79, 0x89, 0x3b, 0x0d, 0x9f, 0xc9, 0x45, 0xd6, 0x92, 0x41, 0xe7, 0x3f, 0x2f, 0xa6, - 0x6d, 0x94, 0xb3, 0x67, 0xde, 0x28, 0xe7, 0x46, 0x6e, 0x94, 0x8f, 0xb2, 0xb9, 0x69, 0x33, 0x57, - 0xfa, 0xfe, 0x24, 0xe4, 0xac, 0xd6, 0x36, 0xf3, 0x5b, 0x4c, 0x98, 0xb0, 0x7d, 0x57, 0x38, 0xd3, - 0xb6, 0xef, 0x12, 0x4b, 0x5a, 0xef, 0x77, 0xf0, 0x0b, 0x61, 0x49, 0x69, 0x81, 0xd8, 0xad, 0x06, - 0x76, 0x7c, 0xbc, 0xe5, 0xf6, 0xd8, 0x01, 0x89, 0x79, 0x99, 0x2a, 0x90, 0x2c, 0x87, 0xed, 0x9d, - 0xf4, 0x89, 0x95, 0xa6, 0x33, 0xc7, 0x5d, 0x4d, 0x19, 0x86, 0x1e, 0xc1, 0x0c, 0x63, 0xea, 0xfa, - 0x81, 0xeb, 0xbd, 0xe4, 0xf6, 0x4d, 0x39, 0xc3, 0x89, 0xde, 0x6d, 0xc8, 0x84, 0xec, 0xe8, 0xa2, - 0xf0, 0xb2, 0x85, 0xfa, 0xda, 0x49, 0xd7, 0x63, 0xcd, 0x4d, 0x89, 0x85, 0x0a, 0x41, 0xe8, 0x4d, - 0x98, 0xff, 0xa8, 0xdc, 0xb0, 0x70, 0xdb, 0x25, 0xb3, 0x51, 0xed, 0x1e, 0x62, 0x3f, 0xe0, 0x51, - 0x89, 0x24, 0x02, 0x7d, 0x0e, 0x96, 0x24, 0x20, 0x6d, 0xb1, 0xe2, 0x9e, 0xf4, 0x03, 0x2a, 0x91, - 0x59, 0x4b, 0x8f, 0x24, 0x0b, 0x23, 0x21, 0x2a, 0xee, 0xf1, 0xa0, 0x87, 0xc9, 0xe6, 0xd7, 0x0f, - 0xbc, 0x2e, 0x66, 0x32, 0x99, 0xb5, 0x86, 0x91, 0x10, 0x75, 0x91, 0xd0, 0x9b, 0x8e, 0x8f, 0xc9, - 0x70, 0x80, 0x32, 0x6a, 0x30, 0x31, 0xfa, 0x86, 0xe3, 0x07, 0x91, 0x9c, 0x6a, 0x30, 0xe8, 0x11, - 0xac, 0x49, 0xd0, 0x1d, 0xaf, 0x7b, 0xd8, 0xed, 0x3b, 0x3d, 0x75, 0x41, 0x67, 0x28, 0xf7, 0x48, - 0x3a, 0x22, 0xb8, 0x9a, 0xa1, 0x50, 0xc1, 0xcd, 0x59, 0x3a, 0xd4, 0xca, 0xbb, 0x30, 0x9f, 0x58, - 0xc8, 0x51, 0xa7, 0xb8, 0xac, 0x7c, 0x8a, 0xfb, 0x18, 0xf2, 0x74, 0xcb, 0x6b, 0xbb, 0x5e, 0x87, - 0x30, 0x92, 0xc1, 0x72, 0x46, 0x32, 0xba, 0x75, 0xc8, 0xda, 0x2f, 0x07, 0x8c, 0x6f, 0x4e, 0x35, - 0x1b, 0x8c, 0x87, 0x60, 0x2d, 0x4a, 0x43, 0xec, 0x2d, 0x35, 0x31, 0x44, 0x7c, 0x67, 0x2c, 0xfa, - 0xbb, 0xf4, 0xd3, 0x0c, 0x00, 0xad, 0x9f, 0x3a, 0x06, 0x84, 0xa4, 0xe9, 0x1c, 0x63, 0x61, 0x92, - 0xc9, 0x6f, 0xd9, 0xe6, 0x67, 0x54, 0x9b, 0xcf, 0xbb, 0x33, 0x11, 0x75, 0xa7, 0x08, 0xd3, 0xdb, - 0xce, 0x8b, 0x56, 0xf7, 0x77, 0xc4, 0x51, 0x4b, 0x14, 0xc9, 0xfe, 0x20, 0xcc, 0x72, 0x95, 0x47, - 0x12, 0x22, 0x00, 0x0d, 0x31, 0x34, 0xeb, 0x55, 0x2e, 0xc5, 0xf4, 0x37, 0xfa, 0x1c, 0x64, 0xec, - 0x16, 0xdf, 0x92, 0x57, 0x36, 0xd8, 0xc5, 0xc2, 0x86, 0xb8, 0x58, 0xd8, 0xb0, 0xc5, 0xc5, 0x02, - 0x3b, 0xa4, 0xfe, 0xc9, 0xbf, 0xdd, 0x34, 0xac, 0x8c, 0xdd, 0x22, 0x9b, 0x64, 0xbd, 0xdf, 0xee, - 0x9d, 0x74, 0x70, 0xed, 0xc5, 0x80, 0x68, 0x02, 0xdf, 0x84, 0xb8, 0x7d, 0xc3, 0xec, 0xa0, 0x93, - 0xb3, 0x46, 0x50, 0x11, 0x87, 0xb8, 0xf6, 0x82, 0x52, 0x6c, 0x39, 0x5e, 0xa7, 0xea, 0x3e, 0xef, - 0x27, 0x2a, 0x62, 0xfb, 0xf5, 0x28, 0xb2, 0x52, 0x09, 0xc0, 0xf6, 0x5d, 0x31, 0xc3, 0x8b, 0x30, - 0xc9, 0xd4, 0x8a, 0x2d, 0x22, 0x2b, 0x94, 0x7e, 0x61, 0x10, 0x2f, 0x8f, 0xee, 0x8f, 0x34, 0xb6, - 0xaa, 0xdd, 0x1b, 0xef, 0x41, 0x7e, 0x67, 0x20, 0xbc, 0xf1, 0x4c, 0x32, 0x0e, 0x56, 0x69, 0x52, - 0xde, 0x9d, 0x81, 0x15, 0xd1, 0xa1, 0xcd, 0xf0, 0x86, 0x81, 0x6d, 0x94, 0xb7, 0x34, 0x37, 0x0c, - 0x94, 0x20, 0xfd, 0x9a, 0xe1, 0xa2, 0x23, 0xc5, 0xa5, 0x06, 0x14, 0x2a, 0xcd, 0xe8, 0xfc, 0xa8, - 0x1b, 0xeb, 0x1b, 0x22, 0xde, 0x97, 0x49, 0xbf, 0xd5, 0x60, 0x14, 0xa5, 0x9f, 0xf1, 0xb9, 0x73, - 0x82, 0x21, 0x73, 0x37, 0x7e, 0x7d, 0xa3, 0x67, 0x4c, 0x34, 0xf4, 0x2b, 0x9c, 0xb1, 0x6f, 0xe6, - 0x60, 0x5a, 0x48, 0x90, 0xe2, 0xd8, 0x1b, 0xc2, 0xd3, 0xe2, 0x00, 0xb4, 0x01, 0x53, 0xdb, 0x38, - 0x78, 0xea, 0x76, 0x74, 0x26, 0x81, 0x61, 0xa8, 0x49, 0xe0, 0x54, 0xe8, 0xbe, 0xac, 0xff, 0x54, - 0x95, 0x63, 0xde, 0x47, 0x84, 0xe5, 0x63, 0x94, 0xed, 0x45, 0x99, 0x06, 0x94, 0x42, 0x97, 0x8e, - 0x2a, 0x7d, 0xe1, 0xee, 0x8d, 0x78, 0x40, 0x49, 0xf1, 0xfb, 0x2c, 0x85, 0x05, 0x3d, 0x20, 0xc2, - 0x10, 0xd5, 0x30, 0x49, 0x6b, 0xb8, 0xae, 0x91, 0xd2, 0xa8, 0x02, 0x99, 0x81, 0xf0, 0xdb, 0x12, - 0xff, 0x54, 0x92, 0xdf, 0x4e, 0xf0, 0x4b, 0x0c, 0xc4, 0xfd, 0x8a, 0xd4, 0x53, 0x77, 0x02, 0x88, - 0xb0, 0x96, 0xac, 0xc8, 0xf7, 0xd5, 0x73, 0x19, 0x77, 0xdc, 0x8a, 0x6a, 0xc7, 0x23, 0xbc, 0xa5, - 0x9e, 0xe2, 0xee, 0xab, 0xfa, 0xce, 0x2f, 0x01, 0x8a, 0x69, 0xca, 0x69, 0xa9, 0xd6, 0xe1, 0xf3, - 0x8a, 0x02, 0xd1, 0xcd, 0x32, 0xe6, 0x02, 0x4b, 0x68, 0x4b, 0x51, 0xb6, 0xfb, 0xaa, 0xb2, 0xd0, - 0x8d, 0x53, 0xd3, 0xb0, 0xc0, 0x5b, 0xaa, 0x6a, 0xbd, 0x0b, 0xb3, 0x55, 0x4c, 0x36, 0x36, 0xde, - 0x1d, 0x1e, 0xa3, 0xb9, 0x2a, 0xb3, 0x2b, 0x04, 0x96, 0x4a, 0x8f, 0x36, 0x61, 0x6e, 0xd7, 0x73, - 0x5f, 0xbc, 0x8c, 0x16, 0x6c, 0x96, 0x1b, 0x78, 0xa9, 0x06, 0x95, 0xc2, 0x8a, 0x71, 0x90, 0x5d, - 0x38, 0x1e, 0x6b, 0x6d, 0x9e, 0x1c, 0x53, 0x27, 0x30, 0x6b, 0xe9, 0x50, 0x68, 0x53, 0x8a, 0x56, - 0x87, 0xc7, 0xb6, 0xcb, 0xe9, 0xc7, 0x36, 0x2b, 0x49, 0x4e, 0xe7, 0xfc, 0x29, 0x6e, 0x1f, 0x6d, - 0x61, 0xa7, 0x17, 0x3c, 0xa5, 0x51, 0x9d, 0xf8, 0x9c, 0x47, 0x68, 0x4b, 0xa6, 0x45, 0x36, 0x2c, - 0xb6, 0xda, 0x4f, 0x71, 0xe7, 0xa4, 0x87, 0xb9, 0x8b, 0x4a, 0x9d, 0x74, 0x1a, 0xdf, 0x29, 0xdc, - 0x5d, 0x93, 0xeb, 0xd0, 0xd1, 0x59, 0x5a, 0xee, 0x92, 0x0b, 0x05, 0xaa, 0x89, 0xfe, 0xc0, 0xed, - 0xfb, 0x78, 0xc8, 0xd1, 0x8c, 0x6f, 0xd3, 0x19, 0x65, 0x9b, 0x16, 0x8e, 0x13, 0xdb, 0xbc, 0x45, - 0x71, 0xd8, 0x3d, 0x40, 0x69, 0x03, 0x90, 0x24, 0xcf, 0x52, 0xbb, 0xef, 0x77, 0x3d, 0xc9, 0x18, - 0x89, 0x62, 0xe9, 0x3f, 0xb3, 0x34, 0x2c, 0xc7, 0xc8, 0x2e, 0xd6, 0x6a, 0x5d, 0x87, 0x7c, 0xcd, - 0xf3, 0x5c, 0xaf, 0xe2, 0x76, 0x30, 0x1d, 0xc2, 0xac, 0x15, 0x01, 0x88, 0x2b, 0x4e, 0x0b, 0xdb, - 0xd8, 0xf7, 0x9d, 0x43, 0xcc, 0xe3, 0x0c, 0x0a, 0x0c, 0xad, 0x02, 0xd4, 0xfd, 0xad, 0xf2, 0x63, - 0x8c, 0x07, 0xd8, 0xa3, 0x56, 0x27, 0x67, 0x49, 0x10, 0xf4, 0xae, 0x32, 0xbb, 0xdc, 0xac, 0x5c, - 0x49, 0x18, 0x46, 0x86, 0xe6, 0x96, 0x51, 0x59, 0x0f, 0xa2, 0x68, 0xd2, 0x21, 0x86, 0x5b, 0x16, - 0x55, 0xd1, 0x24, 0xbc, 0xa5, 0x50, 0x13, 0x69, 0xa3, 0xb6, 0x86, 0x37, 0x9f, 0x4b, 0x36, 0x2f, - 0xa1, 0x2d, 0x99, 0x96, 0xa8, 0x58, 0xa5, 0x77, 0xe2, 0x07, 0xd8, 0xab, 0x62, 0x72, 0x3a, 0xf5, - 0xb9, 0x71, 0x51, 0x54, 0x4c, 0xa5, 0xb0, 0x62, 0x1c, 0xe8, 0x01, 0xe4, 0xa3, 0x5b, 0x02, 0xd0, - 0x88, 0xa9, 0x40, 0x32, 0x01, 0xc5, 0xfe, 0x49, 0x2f, 0xb0, 0x22, 0x16, 0xf4, 0x00, 0x40, 0x32, - 0x8d, 0xcc, 0xc6, 0xac, 0xca, 0x15, 0x24, 0x05, 0xc9, 0x82, 0x98, 0x79, 0x24, 0x0a, 0x84, 0x3d, - 0x66, 0xe1, 0x66, 0x34, 0x93, 0x27, 0xe1, 0x2d, 0x85, 0xba, 0xf4, 0x88, 0xc6, 0xb6, 0x98, 0xff, - 0x1b, 0x4e, 0xcb, 0x5b, 0x64, 0x07, 0x25, 0x10, 0xbf, 0x68, 0xd0, 0x7d, 0x7d, 0x29, 0xb1, 0x98, - 0x04, 0xcb, 0x97, 0x52, 0xd0, 0x96, 0x5e, 0x55, 0x16, 0x82, 0xb8, 0x6f, 0x1f, 0xd2, 0x7d, 0x9b, - 0xbb, 0x6f, 0xb4, 0x50, 0x7a, 0x08, 0xb3, 0xb6, 0xe3, 0x1f, 0xd9, 0xce, 0x41, 0x0f, 0xef, 0xf9, - 0xd8, 0x23, 0x6a, 0x44, 0xfe, 0xf6, 0x23, 0x5f, 0x3a, 0x2c, 0x13, 0xdc, 0xae, 0xe3, 0xfb, 0xcf, - 0x5d, 0xaf, 0xc3, 0x83, 0x1b, 0x61, 0xb9, 0xf4, 0x0d, 0x83, 0xf4, 0x92, 0xda, 0x2d, 0xad, 0x1b, - 0x93, 0xee, 0x8b, 0x2b, 0xf1, 0x97, 0x89, 0xf8, 0x95, 0x4c, 0x78, 0xe9, 0x97, 0x95, 0x2f, 0xfd, - 0x56, 0xe9, 0xde, 0xaf, 0x3a, 0xe5, 0x12, 0xa4, 0xf4, 0x17, 0x19, 0x22, 0xc3, 0xfd, 0x27, 0xdd, - 0xc3, 0xca, 0x53, 0xa7, 0x7f, 0x88, 0xd1, 0xbd, 0xb0, 0x77, 0xfc, 0xea, 0x68, 0x41, 0x3d, 0x70, - 0x50, 0x54, 0x34, 0x83, 0x6c, 0x1c, 0xf7, 0x01, 0x18, 0xbb, 0x74, 0x50, 0xb9, 0x9e, 0x8c, 0x6f, - 0x44, 0x34, 0x96, 0x44, 0x8f, 0x6c, 0x98, 0xab, 0xf7, 0xbb, 0x41, 0xd7, 0xe9, 0x6d, 0xe3, 0xe3, - 0x03, 0xec, 0x09, 0xaf, 0xec, 0xcd, 0xb4, 0x1a, 0x36, 0x54, 0x72, 0x76, 0x74, 0x8e, 0xd5, 0xb1, - 0x52, 0x86, 0x05, 0x0d, 0xd9, 0xa9, 0xae, 0xd7, 0xde, 0x80, 0xd9, 0xd6, 0xd3, 0x93, 0xa0, 0xe3, - 0x3e, 0xef, 0xb3, 0xad, 0x8d, 0xac, 0x0d, 0x0d, 0x9f, 0x89, 0x25, 0x13, 0xc5, 0xd2, 0xdf, 0x4f, - 0xc2, 0xe5, 0x98, 0x09, 0xd7, 0xae, 0xee, 0x2d, 0x98, 0xdd, 0x74, 0xdd, 0xc0, 0x0f, 0x3c, 0x67, - 0x30, 0xe8, 0xf6, 0x0f, 0x69, 0xa3, 0x39, 0x4b, 0x05, 0x12, 0xd3, 0xc0, 0x63, 0x36, 0x74, 0x42, - 0x27, 0xe8, 0x84, 0x2a, 0xa6, 0x41, 0x42, 0x5b, 0x32, 0x2d, 0xb3, 0x49, 0xd1, 0x54, 0x71, 0x77, - 0xad, 0x98, 0x36, 0x95, 0x96, 0xba, 0xfa, 0xef, 0xc6, 0x46, 0xcc, 0x7d, 0xb5, 0xab, 0xaa, 0x61, - 0x90, 0x08, 0xac, 0xd8, 0x0c, 0x3d, 0x86, 0x79, 0x16, 0x58, 0x93, 0x22, 0x6d, 0xdc, 0xb2, 0x2a, - 0x2e, 0x63, 0x82, 0xc8, 0x4a, 0xf2, 0x25, 0x5d, 0x91, 0xe9, 0x53, 0xba, 0x22, 0x8f, 0x61, 0xfe, - 0x91, 0xdb, 0xed, 0xb3, 0xe8, 0x33, 0xb7, 0x7f, 0xdc, 0xd0, 0x2a, 0xbd, 0x49, 0x10, 0x59, 0x49, - 0x3e, 0xb4, 0x05, 0x26, 0xab, 0x9d, 0xfa, 0x2a, 0xac, 0x43, 0xf9, 0xa4, 0x2b, 0x1a, 0xa7, 0xb1, - 0x12, 0x5c, 0x64, 0x79, 0xcb, 0x9d, 0x8e, 0xd0, 0x42, 0x9d, 0x6f, 0x27, 0xa1, 0x2d, 0x99, 0x96, - 0x58, 0xfe, 0x50, 0x54, 0x18, 0x77, 0x21, 0x69, 0xf9, 0x55, 0x0a, 0x2b, 0xc6, 0x51, 0xc2, 0x7a, - 0x5f, 0x45, 0x2b, 0xaf, 0x31, 0x49, 0xcc, 0x8c, 0x2f, 0x89, 0xa5, 0x3f, 0xcb, 0xc0, 0x72, 0x2c, - 0x5c, 0x67, 0x3b, 0xde, 0x21, 0x0e, 0xe8, 0x45, 0x21, 0x5f, 0x22, 0xd2, 0x08, 0xb3, 0xd6, 0x79, - 0x4b, 0x81, 0xd1, 0x60, 0x9b, 0x4c, 0x93, 0x61, 0x34, 0x32, 0x8c, 0xd8, 0xd9, 0xda, 0x0b, 0x62, - 0x82, 0xba, 0x01, 0xbf, 0x83, 0x0e, 0xcb, 0xc4, 0x85, 0xe4, 0xf5, 0xd9, 0xdd, 0x63, 0xec, 0x9e, - 0x04, 0x76, 0xb7, 0x7d, 0xe4, 0x73, 0xeb, 0xa8, 0x43, 0x11, 0x0e, 0x5b, 0xc3, 0xc1, 0x8c, 0xa6, - 0x0e, 0x85, 0x3e, 0x07, 0x4b, 0x35, 0x62, 0x2e, 0x9c, 0x00, 0x57, 0x4e, 0x3c, 0x0f, 0xf7, 0x03, - 0x4a, 0xe3, 0xf3, 0xdb, 0x08, 0x3d, 0xb2, 0x74, 0x47, 0x23, 0x95, 0x6c, 0x28, 0x5d, 0x9f, 0x5e, - 0xa7, 0xb3, 0xe9, 0x08, 0xcb, 0xa5, 0x9e, 0x46, 0xa9, 0xd0, 0x3d, 0xc8, 0x92, 0xfd, 0x86, 0x5b, - 0x69, 0x45, 0x27, 0x94, 0x8d, 0x8a, 0xdb, 0x6a, 0x4a, 0x4c, 0x27, 0xd5, 0xf1, 0x8f, 0xaa, 0x4e, - 0xe0, 0x1c, 0x38, 0xbe, 0x30, 0x79, 0x0a, 0x8c, 0x58, 0x3d, 0x55, 0x8b, 0xd2, 0xad, 0xde, 0x9b, - 0x49, 0x95, 0x18, 0x42, 0xfd, 0xba, 0x22, 0xf6, 0xe9, 0xde, 0x6c, 0xe9, 0x97, 0x46, 0x5c, 0xca, - 0xcf, 0x7a, 0x2b, 0x81, 0x3e, 0x4c, 0xd9, 0x5b, 0x36, 0xd2, 0xf5, 0x65, 0x9c, 0xdd, 0x85, 0xe8, - 0x0a, 0x59, 0x43, 0x7e, 0xaf, 0x40, 0x7f, 0x5f, 0xc4, 0x8e, 0xf3, 0xe7, 0x19, 0xd5, 0xa5, 0x0c, - 0x13, 0x73, 0x0c, 0x29, 0x31, 0xe7, 0x8b, 0xec, 0x82, 0xda, 0xe9, 0x77, 0x44, 0x8a, 0xd0, 0xb5, - 0x21, 0xe7, 0x0b, 0x71, 0x3f, 0x25, 0x58, 0xc8, 0x54, 0x8a, 0x70, 0x3c, 0x3f, 0x19, 0x88, 0x10, - 0x7c, 0x05, 0x80, 0x53, 0x11, 0x85, 0xcb, 0xd2, 0xaa, 0x6f, 0x0c, 0xa9, 0xba, 0x5e, 0x15, 0xf1, - 0x82, 0x88, 0x0d, 0x7d, 0x04, 0x4b, 0xda, 0xcb, 0x29, 0xbe, 0x95, 0xbc, 0x22, 0xd7, 0xa7, 0x4f, - 0x5d, 0xd4, 0xf3, 0x97, 0xfe, 0x32, 0x93, 0x52, 0x33, 0x11, 0x81, 0x5d, 0x0f, 0x0f, 0x1c, 0x8f, - 0x29, 0x0f, 0x59, 0x91, 0x08, 0x40, 0xc6, 0x5b, 0xeb, 0x13, 0x6d, 0xe8, 0xf0, 0xcd, 0x56, 0x14, - 0x53, 0xf2, 0xa4, 0xee, 0xc2, 0x62, 0x78, 0xc7, 0x4b, 0x33, 0x29, 0x59, 0xb8, 0x9d, 0x2f, 0xb5, - 0x16, 0x87, 0x36, 0x00, 0x69, 0xee, 0xb1, 0x99, 0xe5, 0xd0, 0x60, 0x88, 0x5b, 0x26, 0x5d, 0xbb, - 0xb3, 0x90, 0xa8, 0x04, 0x21, 0x3d, 0x63, 0x17, 0xc8, 0x2c, 0xb9, 0x83, 0x15, 0x88, 0x8d, 0x20, - 0x83, 0x0e, 0x02, 0xdc, 0xe1, 0x21, 0xce, 0xb0, 0x5c, 0xfa, 0x17, 0x43, 0xbd, 0xca, 0x0e, 0x67, - 0x47, 0xd8, 0x5c, 0xd9, 0x56, 0x1a, 0xe3, 0xd9, 0xca, 0x4c, 0xba, 0xad, 0x7c, 0x1b, 0x96, 0x23, - 0x9d, 0x57, 0x98, 0xd8, 0x5c, 0xa6, 0x60, 0xd3, 0x2d, 0x66, 0x76, 0x98, 0xc5, 0xfc, 0x57, 0x80, - 0x02, 0xef, 0x05, 0x3d, 0x7b, 0x88, 0xf4, 0x41, 0x43, 0x4a, 0x1f, 0xfc, 0xbf, 0x95, 0xba, 0x53, - 0x0e, 0x23, 0x94, 0x39, 0xaa, 0x86, 0xaf, 0x6a, 0xc2, 0x46, 0x34, 0x15, 0x6e, 0xcc, 0xcc, 0xf1, - 0xfc, 0x99, 0x32, 0xc7, 0x41, 0x9f, 0x30, 0xa4, 0x5e, 0xf1, 0x17, 0xc6, 0x49, 0x05, 0x9a, 0x19, - 0x99, 0x0a, 0x34, 0x7b, 0xa6, 0x54, 0xa0, 0xb9, 0x33, 0xe5, 0xa0, 0x5f, 0x1e, 0x27, 0x07, 0xdd, - 0x1c, 0x2f, 0x45, 0x68, 0x3e, 0x96, 0x22, 0x34, 0xe2, 0x1e, 0x13, 0x5d, 0x44, 0xc2, 0xcf, 0xc2, - 0x45, 0x25, 0xfc, 0x2c, 0x5e, 0x40, 0xc2, 0xcf, 0xd2, 0xf9, 0x13, 0x7e, 0x96, 0x2f, 0x24, 0xe1, - 0xe7, 0xca, 0x05, 0x24, 0xfc, 0x14, 0xc7, 0x48, 0xf8, 0x19, 0x9e, 0x95, 0x7f, 0x75, 0x54, 0x56, - 0xfe, 0xd0, 0x84, 0xa1, 0x95, 0xe1, 0x09, 0x43, 0x9f, 0x50, 0x46, 0xff, 0x77, 0x0c, 0xf6, 0x44, - 0x87, 0xbf, 0x57, 0xe1, 0x26, 0xd9, 0xd0, 0xbf, 0x57, 0x71, 0x02, 0xbc, 0xc1, 0x28, 0x14, 0xab, - 0xc3, 0x40, 0x2b, 0x16, 0x14, 0x24, 0xa4, 0xa6, 0x83, 0x9f, 0x51, 0x3b, 0x78, 0x25, 0xc5, 0xb0, - 0xc9, 0x5e, 0xd1, 0x0f, 0xb3, 0x34, 0x45, 0xe5, 0x42, 0xcc, 0xff, 0x6f, 0xb2, 0x4a, 0x7e, 0x8d, - 0xb3, 0x4a, 0x46, 0xd8, 0xd6, 0xd9, 0x71, 0x73, 0x44, 0x88, 0xbc, 0xdb, 0xe3, 0xc8, 0xbb, 0xfd, - 0xc9, 0xca, 0xbb, 0xad, 0x97, 0xf7, 0xbf, 0x9d, 0x00, 0x90, 0x4e, 0x54, 0xba, 0x73, 0xb9, 0x50, - 0x81, 0x8c, 0xa4, 0x02, 0xb7, 0x60, 0x96, 0xa8, 0x37, 0xee, 0xab, 0xce, 0x8d, 0x0a, 0x8c, 0x49, - 0x4d, 0x76, 0x6c, 0xa9, 0x19, 0xbd, 0x2b, 0x4d, 0x5e, 0xd4, 0xae, 0x34, 0x75, 0x01, 0xbb, 0xd2, - 0xf4, 0xf9, 0xde, 0x56, 0xe5, 0x46, 0x59, 0xf1, 0xd2, 0xdf, 0x18, 0xe1, 0x22, 0x11, 0x31, 0x7a, - 0x2f, 0x26, 0x46, 0xa5, 0xc4, 0x6d, 0xd7, 0x28, 0x49, 0xfa, 0x60, 0x94, 0x24, 0xbd, 0xa9, 0x4a, - 0xd2, 0xb2, 0xa6, 0x05, 0xd7, 0xc3, 0xb2, 0x20, 0xfd, 0x38, 0x13, 0xbf, 0x8b, 0x4b, 0x0b, 0x4a, - 0xaa, 0x82, 0x93, 0x19, 0x2d, 0x38, 0x13, 0x17, 0x28, 0x38, 0xd9, 0x8b, 0x12, 0x9c, 0xc9, 0x0b, - 0x10, 0x9c, 0xa9, 0x11, 0x82, 0x53, 0xfa, 0xd6, 0x44, 0xfc, 0xf6, 0x05, 0xbd, 0x05, 0x39, 0xae, - 0xca, 0x62, 0xf9, 0x17, 0x34, 0x6a, 0x2e, 0x1c, 0x52, 0x41, 0x4a, 0xd8, 0x2a, 0x82, 0x2d, 0x93, - 0x64, 0xab, 0xa8, 0x6c, 0x82, 0x14, 0xbd, 0x43, 0xf3, 0x85, 0x38, 0x1f, 0xdb, 0xc5, 0x16, 0x75, - 0xd7, 0xf1, 0x9c, 0x31, 0x22, 0x46, 0x0f, 0xa0, 0x10, 0x09, 0x8a, 0x38, 0xe1, 0xa7, 0xc8, 0x91, - 0xb8, 0xf0, 0x92, 0x18, 0x50, 0x55, 0x84, 0x86, 0x3a, 0xbc, 0x06, 0x96, 0xdd, 0x56, 0x4c, 0xc6, - 0x3f, 0x3b, 0x72, 0x1d, 0x2a, 0x53, 0x7a, 0x84, 0x60, 0xea, 0x9c, 0x11, 0x82, 0x3f, 0x30, 0xa0, - 0xc0, 0x57, 0x86, 0xfa, 0x09, 0x9f, 0xa7, 0xcb, 0xc2, 0x76, 0x7b, 0x83, 0xef, 0xf6, 0xa1, 0x3b, - 0xc4, 0x31, 0xca, 0x95, 0x50, 0x48, 0x8e, 0xee, 0xb3, 0x39, 0x66, 0xbc, 0x19, 0x3e, 0xca, 0xc8, - 0x95, 0x12, 0xb1, 0x59, 0x99, 0x39, 0x62, 0x28, 0x7d, 0x37, 0x0b, 0x4b, 0x3c, 0x14, 0x24, 0x02, - 0xca, 0x3c, 0xa5, 0xe0, 0x35, 0x98, 0x6b, 0x9e, 0x1c, 0xef, 0x3c, 0x89, 0x2a, 0x67, 0x4e, 0x4c, - 0x0c, 0x4a, 0x54, 0x92, 0x42, 0xc2, 0xfe, 0x33, 0x43, 0xaf, 0x02, 0xd1, 0x3a, 0x98, 0x82, 0x2f, - 0x4c, 0xa8, 0x66, 0xe7, 0xef, 0x04, 0x9c, 0x9c, 0x7e, 0x9a, 0xf8, 0x45, 0x10, 0x5e, 0xfa, 0xf2, - 0x12, 0xb2, 0xa1, 0xc0, 0x7e, 0x6d, 0xbe, 0x7c, 0x8c, 0x45, 0xbe, 0xe2, 0x5d, 0x79, 0x0d, 0xb4, - 0x23, 0xd9, 0x90, 0x98, 0x58, 0x88, 0x4c, 0xae, 0x06, 0x3d, 0xd1, 0x5d, 0xc7, 0xb3, 0xc7, 0x63, - 0xef, 0x8c, 0x51, 0x77, 0x9c, 0x35, 0xfe, 0x8a, 0x2c, 0xbc, 0xb2, 0xa7, 0x3e, 0xd3, 0xa1, 0xb8, - 0x43, 0xe0, 0xa9, 0x79, 0xe2, 0x5c, 0x9d, 0xc4, 0xac, 0x3c, 0x00, 0x33, 0xde, 0x71, 0x7d, 0xba, - 0xbd, 0x3e, 0x57, 0x4f, 0x79, 0x04, 0xa6, 0x74, 0x6e, 0x54, 0x2d, 0x4a, 0x98, 0xef, 0xbf, 0x0c, - 0xb8, 0x6a, 0x61, 0x9f, 0xc5, 0x45, 0x3f, 0x72, 0x02, 0xec, 0x1d, 0x3b, 0xde, 0x91, 0x90, 0x91, - 0x68, 0xa5, 0x0c, 0x65, 0xa5, 0xbe, 0xa4, 0xae, 0x54, 0x26, 0xf9, 0x48, 0x2e, 0xb5, 0xce, 0x11, - 0xab, 0xa5, 0x9f, 0xc5, 0x89, 0x4f, 0x6a, 0x16, 0x4b, 0xdf, 0xe3, 0xef, 0x21, 0x87, 0x7a, 0xf4, - 0xbf, 0x79, 0x95, 0xf0, 0xff, 0xf9, 0x55, 0xc2, 0x3f, 0x88, 0xa7, 0xc6, 0xc4, 0xb9, 0x7a, 0x10, - 0x1e, 0xda, 0x98, 0x19, 0x5f, 0x4b, 0x6c, 0x77, 0xd4, 0xb5, 0xa2, 0x24, 0xaa, 0x6b, 0xc5, 0xec, - 0xe4, 0x83, 0xd0, 0x39, 0xcb, 0x0c, 0xe3, 0x4f, 0x75, 0xcd, 0x5a, 0x50, 0x90, 0x2a, 0xd7, 0x44, - 0xf4, 0x37, 0x54, 0xd7, 0x2c, 0xf5, 0xb9, 0xa5, 0x6c, 0x4a, 0x5a, 0xa3, 0xfc, 0xbd, 0x51, 0x95, - 0xea, 0x8e, 0x0e, 0xff, 0x91, 0x53, 0xd3, 0x2a, 0xb4, 0x9a, 0xf5, 0xae, 0xb2, 0x4d, 0x6a, 0x0f, - 0xe2, 0x11, 0x5a, 0xf8, 0x01, 0xf2, 0xc6, 0x7a, 0x2f, 0x3c, 0x3e, 0x71, 0x3f, 0x70, 0x41, 0x73, - 0x68, 0x12, 0x49, 0x02, 0xe2, 0xa0, 0xf5, 0x76, 0xb4, 0xa0, 0xfc, 0xd8, 0xb1, 0xa8, 0x5b, 0x86, - 0x48, 0x72, 0xf9, 0xe2, 0xdf, 0x0b, 0x63, 0x13, 0xfc, 0x0a, 0x61, 0x41, 0x13, 0x91, 0x10, 0x8d, - 0x89, 0x28, 0xc6, 0x1d, 0x91, 0x0c, 0xca, 0xc2, 0xb2, 0xca, 0xf5, 0x98, 0x48, 0x00, 0x52, 0x52, - 0x42, 0x9b, 0x5c, 0x7f, 0xf9, 0x0d, 0x07, 0x4f, 0x4a, 0x99, 0xa6, 0xdc, 0xab, 0xf1, 0xcb, 0x35, - 0x95, 0xca, 0xd2, 0x70, 0xa2, 0x5a, 0x2c, 0x5f, 0x84, 0x2b, 0xeb, 0xc8, 0x7b, 0xba, 0x58, 0x96, - 0x89, 0xd8, 0x0b, 0x3a, 0x3c, 0xcf, 0x9e, 0x97, 0xd0, 0x63, 0x75, 0x2f, 0x00, 0x2a, 0xd6, 0x6f, - 0xa4, 0x25, 0xcf, 0x8c, 0x30, 0xff, 0xf7, 0xe5, 0x93, 0x0c, 0xbf, 0x50, 0x5e, 0xd6, 0x9f, 0x5f, - 0xc4, 0x85, 0x8f, 0x74, 0xf2, 0x91, 0xc3, 0xb1, 0x33, 0xa7, 0x7b, 0x99, 0xa9, 0xcb, 0xf1, 0x9b, - 0x4d, 0xcf, 0xf1, 0xdb, 0xd2, 0x39, 0x15, 0x73, 0x23, 0x8d, 0xa0, 0xc6, 0x6d, 0xb8, 0x0f, 0x57, - 0x93, 0xdb, 0xda, 0x2e, 0xee, 0x77, 0xba, 0xfd, 0x43, 0x1a, 0x1d, 0xce, 0x59, 0xe9, 0x04, 0x68, - 0x13, 0xae, 0x2b, 0x5b, 0x2c, 0xdd, 0x74, 0xa5, 0x63, 0x08, 0x7b, 0x0e, 0x3a, 0x94, 0x86, 0x1c, - 0x3f, 0x35, 0x0d, 0xd0, 0x4b, 0xab, 0xf0, 0x59, 0xe8, 0x10, 0x0a, 0xf4, 0x1e, 0x5c, 0x4b, 0x62, - 0xc3, 0x97, 0x17, 0x22, 0xcc, 0x3c, 0x84, 0xe4, 0xdc, 0x9b, 0xf8, 0x1f, 0x1b, 0x30, 0x23, 0x3b, - 0xf6, 0xda, 0xa3, 0xe5, 0x75, 0xc8, 0xb3, 0x4b, 0x20, 0x91, 0x3d, 0x90, 0xb7, 0x22, 0x00, 0x2a, - 0xc2, 0xb4, 0xba, 0x73, 0x8b, 0xa2, 0x14, 0xab, 0xcf, 0x2a, 0xb1, 0xfa, 0x15, 0xc8, 0x55, 0xdd, - 0xe7, 0x7d, 0x8a, 0x99, 0xa4, 0x98, 0xb0, 0x5c, 0xfa, 0x56, 0x09, 0x4c, 0xa1, 0xdb, 0xe1, 0x0b, - 0xa0, 0xf0, 0xbd, 0x8f, 0x21, 0xbf, 0xf7, 0xd1, 0x85, 0x4f, 0x22, 0xb7, 0x6b, 0x42, 0x71, 0xbb, - 0x76, 0x54, 0x55, 0x63, 0x87, 0xa6, 0xcf, 0xe8, 0x0c, 0x4a, 0xf8, 0xb0, 0x67, 0xb8, 0xba, 0xe9, - 0x3e, 0xb6, 0xf0, 0x3f, 0x6e, 0xaf, 0x3a, 0x60, 0xc6, 0x6e, 0x77, 0xc5, 0xd5, 0xd3, 0xdd, 0xa1, - 0x43, 0x8d, 0x33, 0xc9, 0xdb, 0x67, 0xa2, 0x46, 0x54, 0x97, 0x8f, 0x55, 0xec, 0x7b, 0x48, 0x9f, - 0x1e, 0x5a, 0x7d, 0x48, 0xcd, 0xe6, 0x31, 0xe2, 0x96, 0xb7, 0x05, 0x18, 0x7b, 0x5b, 0x90, 0x36, - 0xae, 0xc2, 0x99, 0x36, 0xae, 0x99, 0x53, 0x6c, 0x5c, 0xb1, 0x6d, 0x76, 0xf6, 0xd4, 0xdb, 0x6c, - 0x62, 0x0f, 0x99, 0x3b, 0xd3, 0x1e, 0xa2, 0x9a, 0xf7, 0xcb, 0xa7, 0x34, 0xef, 0x89, 0x33, 0xbf, - 0x79, 0x96, 0x33, 0x7f, 0x8a, 0xb1, 0x9f, 0x3f, 0xa5, 0xb1, 0x47, 0x17, 0x6e, 0xec, 0x17, 0xce, - 0x6b, 0xec, 0x17, 0xcf, 0x6d, 0xec, 0x97, 0xce, 0x6b, 0xec, 0x97, 0x47, 0x1a, 0x7b, 0xf4, 0x76, - 0x22, 0x17, 0x4b, 0xe4, 0x44, 0xb0, 0x5b, 0xb3, 0x14, 0xac, 0xe6, 0xd8, 0x10, 0x65, 0x5a, 0x14, - 0xb5, 0xc7, 0x86, 0x28, 0xf1, 0xe2, 0xab, 0xb0, 0x18, 0xc3, 0x89, 0x1b, 0xb2, 0xc4, 0xc1, 0x35, - 0xa1, 0xf7, 0x3a, 0x46, 0x66, 0x02, 0xb4, 0x75, 0xa2, 0x41, 0x62, 0x7c, 0x95, 0xa6, 0xb8, 0x51, - 0x4b, 0x04, 0x1d, 0x46, 0xb5, 0xc6, 0x59, 0x59, 0x7b, 0x29, 0xf5, 0x6a, 0x5a, 0xb4, 0x79, 0x8b, - 0xd7, 0x4e, 0xdf, 0xa2, 0x3d, 0xac, 0x45, 0x8e, 0x44, 0x5b, 0x70, 0x33, 0x86, 0xe1, 0x89, 0x3b, - 0x7e, 0xd9, 0xf7, 0xbb, 0x87, 0x7d, 0xdc, 0x29, 0x5e, 0x67, 0xef, 0xd5, 0x46, 0x90, 0xa1, 0x06, - 0xbc, 0x12, 0x1f, 0x55, 0x98, 0xc0, 0x13, 0xd6, 0x75, 0x83, 0xd6, 0x35, 0x9a, 0x30, 0xf5, 0x78, - 0x18, 0x49, 0xca, 0xea, 0x90, 0xe3, 0x61, 0x24, 0x2f, 0x9b, 0x29, 0x19, 0x2c, 0x42, 0x52, 0x6f, - 0x26, 0xef, 0x77, 0xe3, 0x34, 0xa9, 0x51, 0x7d, 0x16, 0xdb, 0x5d, 0xa3, 0xba, 0x3a, 0x84, 0x02, - 0x3d, 0x82, 0x35, 0xed, 0xdd, 0xaf, 0x9c, 0x08, 0xf4, 0x0a, 0xed, 0xc7, 0x48, 0xba, 0x31, 0xee, - 0xbd, 0x4b, 0x63, 0xdd, 0x7b, 0x7f, 0xdd, 0x80, 0x1b, 0xda, 0x2e, 0xd3, 0x90, 0x04, 0x91, 0xb8, - 0x57, 0xa9, 0xc4, 0xbd, 0x3b, 0x54, 0xe2, 0x86, 0xd6, 0xc0, 0x04, 0x6f, 0x78, 0x2b, 0xe8, 0xf7, - 0xd2, 0x52, 0x8c, 0x84, 0xaa, 0xdd, 0xa2, 0xdd, 0x78, 0x70, 0xfa, 0x6e, 0x28, 0x0a, 0x37, 0xb4, - 0x0d, 0xf4, 0x0d, 0x23, 0xe5, 0x1a, 0x80, 0x6e, 0x59, 0xac, 0x1f, 0x9f, 0xa2, 0xfd, 0x28, 0x9f, - 0xbe, 0x1f, 0x51, 0x1d, 0xac, 0x2b, 0xa3, 0x5a, 0x42, 0x7f, 0x68, 0xa4, 0xc8, 0x7e, 0xa5, 0xc9, - 0xf3, 0xae, 0x8a, 0xaf, 0xd1, 0xce, 0xbc, 0x77, 0x96, 0x49, 0xe1, 0x55, 0xb0, 0xbe, 0x8c, 0x68, - 0x07, 0x7d, 0xd3, 0x80, 0x57, 0xd2, 0xbb, 0x2b, 0x7a, 0xf3, 0x3a, 0xed, 0x4d, 0xe5, 0x8c, 0x53, - 0xa3, 0x74, 0x68, 0x74, 0x6b, 0xa9, 0x1a, 0x2d, 0x36, 0xdf, 0xdb, 0x43, 0x34, 0x5a, 0xec, 0xbf, - 0x7f, 0x6a, 0x40, 0x69, 0xe8, 0xd0, 0x59, 0xda, 0xd9, 0x1b, 0x74, 0x60, 0xd5, 0xb3, 0x4f, 0x33, - 0xad, 0x86, 0x8d, 0x6c, 0x8c, 0xf6, 0xd0, 0x77, 0x0d, 0xf8, 0xd4, 0xa8, 0x09, 0x60, 0x3d, 0x5b, - 0xa7, 0x3d, 0x7b, 0x78, 0xae, 0x29, 0x97, 0x3a, 0x37, 0x5e, 0xab, 0xe7, 0x0e, 0x74, 0x7f, 0x0c, - 0x4b, 0x5a, 0xd7, 0xfe, 0x94, 0x71, 0x2a, 0xe5, 0xfd, 0x93, 0x54, 0xfd, 0x7d, 0xfa, 0xe1, 0xb2, - 0x94, 0xa0, 0xda, 0xc8, 0xce, 0x3d, 0x84, 0xab, 0xa9, 0x0e, 0xc2, 0xa8, 0x8a, 0x72, 0x72, 0x45, - 0xf5, 0x44, 0x42, 0x81, 0x6c, 0x8a, 0xce, 0x59, 0x95, 0x7d, 0xd6, 0xaa, 0x76, 0x53, 0x24, 0x5e, - 0xb1, 0xd6, 0xa7, 0xaa, 0x71, 0x27, 0xc5, 0x36, 0x9c, 0x79, 0xb4, 0x16, 0xdc, 0x1a, 0xc7, 0x82, - 0x9e, 0xaa, 0xce, 0x0f, 0xe0, 0xd5, 0x31, 0x0c, 0xe1, 0xa9, 0x04, 0xc5, 0x86, 0xd7, 0xc6, 0xb3, - 0x66, 0xa7, 0xaa, 0x75, 0x0f, 0x5e, 0x1f, 0xd3, 0x94, 0x9c, 0xaa, 0xda, 0x2f, 0xc1, 0xfa, 0xf8, - 0x76, 0xe0, 0x54, 0xa1, 0x9a, 0x3a, 0xcb, 0xcd, 0x11, 0xdf, 0x08, 0x3c, 0xc7, 0x57, 0x79, 0x4a, - 0x7f, 0x95, 0x81, 0x45, 0xdd, 0xd3, 0xc0, 0x21, 0x19, 0xfa, 0xbb, 0x89, 0x4f, 0x5a, 0x6e, 0x8c, - 0x7a, 0x68, 0xa8, 0x7e, 0xda, 0x32, 0x71, 0xd9, 0x72, 0x21, 0x1f, 0xb8, 0x5c, 0xb1, 0x47, 0x7f, - 0x0d, 0x72, 0x58, 0xf2, 0x8e, 0x34, 0xa3, 0xf2, 0x5c, 0x7f, 0xdf, 0x00, 0xd8, 0x74, 0xda, 0x47, - 0x27, 0x03, 0x7a, 0x5f, 0x93, 0x76, 0x99, 0x57, 0xd7, 0x5d, 0xe6, 0xbd, 0xae, 0xbc, 0x4a, 0x08, - 0x2b, 0x19, 0x1e, 0x4f, 0x3a, 0x77, 0x20, 0xef, 0xf7, 0x0d, 0x71, 0x17, 0x55, 0x0f, 0xf0, 0xb1, - 0xf6, 0x03, 0x21, 0x25, 0x98, 0xe1, 0x19, 0xd9, 0x1f, 0x4a, 0x37, 0x9a, 0x0a, 0x8c, 0xd0, 0x54, - 0xf1, 0x13, 0xe7, 0xa4, 0xc7, 0x69, 0x58, 0x44, 0x4f, 0x81, 0x91, 0x25, 0xaa, 0xf7, 0x03, 0xec, - 0xf5, 0x9d, 0x1e, 0xbf, 0x84, 0x0b, 0xcb, 0xa5, 0xbf, 0x36, 0xe4, 0x2b, 0x31, 0xf4, 0x45, 0x98, - 0xae, 0xb8, 0xfd, 0x00, 0xd3, 0xef, 0x68, 0x24, 0x53, 0xa0, 0x43, 0xc2, 0x0d, 0x4e, 0xc5, 0x26, - 0x46, 0xf0, 0xac, 0x58, 0xf4, 0x1d, 0x5c, 0x88, 0x38, 0x65, 0x3a, 0x4d, 0x34, 0x1d, 0xf2, 0x44, - 0xfd, 0x6e, 0x74, 0xf7, 0x86, 0xde, 0x8a, 0x5e, 0x89, 0x26, 0xb2, 0xc6, 0x04, 0xd1, 0x06, 0xa5, - 0x60, 0x1d, 0x63, 0xd4, 0x2b, 0xef, 0x00, 0x44, 0xc0, 0x53, 0xdd, 0x19, 0xbf, 0xae, 0x3c, 0x4e, - 0x1f, 0xf2, 0x7a, 0xe6, 0x63, 0x98, 0x4f, 0xbc, 0xd3, 0x40, 0xb7, 0x60, 0x96, 0x7d, 0xef, 0x46, - 0x3c, 0xfd, 0x60, 0x4c, 0x2a, 0x90, 0x2e, 0x33, 0x67, 0x91, 0xbe, 0x91, 0xa4, 0xc0, 0xd6, 0x9f, - 0x03, 0xec, 0x0d, 0x3a, 0x4e, 0xc0, 0x22, 0xb8, 0x57, 0x60, 0x41, 0xf9, 0x7e, 0x0e, 0x43, 0x99, - 0x97, 0xd0, 0x12, 0xcc, 0x8b, 0xef, 0x22, 0x35, 0x5a, 0x4d, 0x0e, 0x36, 0xd0, 0x02, 0x5c, 0xde, - 0xf3, 0xb1, 0x47, 0x87, 0xcf, 0x81, 0x19, 0x34, 0x0b, 0x79, 0xbb, 0xb5, 0xc3, 0x8b, 0x13, 0x84, - 0x35, 0xfc, 0xc6, 0x51, 0xc8, 0x9a, 0x5d, 0xdf, 0x80, 0x7c, 0xf8, 0x19, 0x60, 0x74, 0x19, 0x0a, - 0x4d, 0xd7, 0x3b, 0x76, 0x7a, 0xb4, 0x68, 0x5e, 0x42, 0x26, 0xcc, 0xf0, 0x87, 0x06, 0x0c, 0x62, - 0xac, 0xff, 0x22, 0x0b, 0x10, 0xbd, 0x2b, 0x47, 0x73, 0x00, 0x76, 0x6b, 0x67, 0x7f, 0x6f, 0xb7, - 0x5a, 0xb6, 0x6b, 0xe6, 0x25, 0x04, 0x30, 0x55, 0xde, 0xdd, 0xad, 0x35, 0xab, 0xa6, 0x81, 0x72, - 0x90, 0xb5, 0x6a, 0xe5, 0xaa, 0x99, 0x41, 0x33, 0x90, 0xb3, 0xad, 0xbd, 0x66, 0x85, 0xd0, 0x4c, - 0x90, 0x4a, 0x1f, 0xd6, 0xec, 0xfd, 0x10, 0x92, 0x45, 0x05, 0x98, 0xae, 0xec, 0x34, 0x9b, 0xb5, - 0x8a, 0x6d, 0x4e, 0x92, 0x2a, 0x79, 0x61, 0xdf, 0xda, 0x31, 0xa7, 0xd0, 0x3c, 0xcc, 0x36, 0x76, - 0x1e, 0xee, 0x6f, 0xd5, 0xca, 0x96, 0xbd, 0x59, 0x2b, 0xdb, 0xe6, 0x34, 0xa9, 0xa1, 0xd2, 0x94, - 0x20, 0x39, 0xda, 0x51, 0x19, 0x92, 0x47, 0x08, 0xe6, 0x2a, 0x5b, 0xb5, 0xca, 0xe3, 0xfd, 0xad, - 0xf2, 0xe3, 0x5a, 0x6d, 0xb7, 0x66, 0x99, 0x40, 0xe6, 0x95, 0xb4, 0x5c, 0x69, 0xec, 0xb5, 0xec, - 0x9a, 0xb5, 0x5f, 0xad, 0xd9, 0xe5, 0x7a, 0xa3, 0x65, 0x16, 0x08, 0x31, 0x41, 0xb4, 0xb6, 0xca, - 0x56, 0x75, 0xbf, 0xde, 0x7c, 0x7f, 0xc7, 0x9c, 0xa1, 0x15, 0x34, 0xf7, 0xcb, 0x8d, 0xc6, 0x0e, - 0xe9, 0xe5, 0x7e, 0xbd, 0x6a, 0xce, 0x92, 0x49, 0x94, 0x2b, 0x68, 0xd9, 0xa4, 0xff, 0x73, 0x74, - 0xfe, 0xe9, 0x0c, 0xec, 0x57, 0x9a, 0xfb, 0x8d, 0xf2, 0x66, 0xad, 0x61, 0x5e, 0x46, 0x45, 0x58, - 0x8c, 0x80, 0x1f, 0xed, 0x58, 0x8f, 0x39, 0xb9, 0x49, 0x6a, 0xde, 0x2d, 0xdb, 0x95, 0x2d, 0x82, - 0x68, 0xd9, 0x3b, 0x56, 0xcd, 0x9c, 0x27, 0x55, 0x54, 0x6b, 0x8d, 0x1a, 0xa3, 0x66, 0x40, 0x44, - 0x80, 0xbb, 0xd6, 0xce, 0x97, 0xbe, 0x2c, 0x0d, 0x6c, 0x01, 0xbd, 0x02, 0x37, 0x78, 0xbd, 0xcd, - 0x9d, 0xe6, 0xfe, 0x87, 0x3b, 0x76, 0xbd, 0xf9, 0x70, 0xdf, 0xaa, 0xed, 0x36, 0xea, 0x95, 0xf2, - 0x7e, 0x73, 0x6f, 0xdb, 0x5c, 0x44, 0xab, 0xb0, 0x92, 0x24, 0x21, 0xe3, 0x68, 0xd4, 0xed, 0x2f, - 0x9b, 0x4b, 0x68, 0x11, 0xcc, 0x56, 0xcd, 0xde, 0xb7, 0x6a, 0x1f, 0xec, 0xd5, 0xad, 0x5a, 0x75, - 0xbf, 0xd1, 0x6a, 0x9a, 0xcb, 0x04, 0xfa, 0x30, 0x0e, 0xbd, 0x22, 0xa6, 0xa6, 0x51, 0xb6, 0x6b, - 0x2d, 0x9b, 0xc2, 0x8a, 0x64, 0x49, 0x28, 0xac, 0x56, 0xae, 0xd6, 0x2c, 0x32, 0x33, 0x57, 0xe9, - 0x92, 0xb0, 0xe9, 0xae, 0x95, 0x1b, 0xf6, 0x96, 0xb9, 0x42, 0x16, 0x9d, 0x2c, 0x3f, 0x65, 0xb9, - 0x86, 0xae, 0xc2, 0x12, 0xef, 0x52, 0xa3, 0x56, 0x6e, 0xd5, 0xb6, 0x76, 0x1a, 0x9c, 0xf5, 0x3a, - 0x41, 0xd1, 0xc9, 0xaf, 0x6c, 0xd5, 0xaa, 0x7b, 0x8d, 0xda, 0x7e, 0x65, 0x67, 0x7b, 0xbb, 0xdc, - 0xac, 0xb6, 0xcc, 0x1b, 0xeb, 0x4d, 0x80, 0xe8, 0x6b, 0x4c, 0x44, 0x32, 0x88, 0x98, 0x33, 0x88, - 0x79, 0x89, 0xb4, 0x20, 0xec, 0x9c, 0x69, 0x10, 0xe1, 0xa5, 0x4a, 0x13, 0x2a, 0xc0, 0x3c, 0xff, - 0xfc, 0x98, 0x85, 0xbf, 0x8a, 0xdb, 0x01, 0xee, 0x98, 0x13, 0xeb, 0xeb, 0x90, 0x0f, 0x3f, 0xf6, - 0x43, 0xd8, 0x5b, 0x38, 0xa0, 0x25, 0xf3, 0x12, 0x61, 0x67, 0xc1, 0x55, 0x06, 0x30, 0xd6, 0xff, - 0x29, 0x0b, 0x48, 0x1c, 0x29, 0x24, 0xdd, 0x24, 0x12, 0xdf, 0x6d, 0x1f, 0xc9, 0x2a, 0x29, 0x7d, - 0x55, 0x25, 0x54, 0x49, 0xa2, 0xa9, 0x09, 0x70, 0x06, 0x2d, 0xd3, 0x9c, 0x90, 0x38, 0x7c, 0x82, - 0xb4, 0xfe, 0x10, 0x07, 0xa1, 0xa6, 0x67, 0xc9, 0xa4, 0xc4, 0xec, 0x0d, 0x47, 0x4d, 0x92, 0x15, - 0x69, 0x61, 0xa6, 0x90, 0x1c, 0x36, 0x45, 0x84, 0x4d, 0x4d, 0xfa, 0xe1, 0x98, 0x69, 0x74, 0x13, - 0xae, 0xb5, 0x70, 0x90, 0xbc, 0x9b, 0xe0, 0x04, 0x39, 0xb4, 0x02, 0xcb, 0x9c, 0x20, 0x0c, 0x6e, - 0x73, 0x5c, 0x9e, 0x4c, 0x21, 0xfb, 0xcd, 0x67, 0xcd, 0x04, 0x32, 0x30, 0x01, 0x0a, 0x9f, 0xb8, - 0x98, 0x05, 0xb2, 0xfe, 0xbb, 0xc4, 0xde, 0xf1, 0x7c, 0x3a, 0x73, 0x86, 0xf0, 0x5a, 0xf8, 0xd8, - 0x7d, 0x26, 0x5e, 0x3c, 0x9a, 0xb3, 0xa4, 0x97, 0x6a, 0xe2, 0x24, 0x6f, 0x68, 0x0e, 0xdd, 0x80, - 0xab, 0xec, 0xb7, 0x26, 0x68, 0x6d, 0x5e, 0x46, 0xd7, 0xe0, 0x4a, 0x0c, 0x2d, 0x76, 0x03, 0xa6, - 0x4e, 0xe2, 0xd4, 0xc3, 0xeb, 0x9b, 0x47, 0xd7, 0xa1, 0x98, 0x4c, 0xdb, 0xe1, 0x58, 0x84, 0x6e, - 0xc1, 0x9a, 0x08, 0xe2, 0x26, 0xc3, 0xbb, 0x9c, 0x6a, 0x81, 0xcc, 0x1c, 0x0b, 0x80, 0xc5, 0x4e, - 0x20, 0x9c, 0x60, 0x11, 0x7d, 0x0a, 0x5e, 0x61, 0x04, 0x5a, 0x07, 0x93, 0x93, 0x2d, 0xad, 0x7f, - 0xdb, 0x80, 0x59, 0xe5, 0xb6, 0x89, 0xe8, 0xb5, 0x00, 0xf0, 0xcc, 0x15, 0xf3, 0x12, 0x59, 0x71, - 0x01, 0x54, 0xde, 0xad, 0x9b, 0x06, 0x69, 0x28, 0x81, 0x12, 0xe7, 0x47, 0x0b, 0xb7, 0x71, 0xf7, - 0x19, 0xee, 0x98, 0x19, 0x32, 0x4b, 0x09, 0xb2, 0xf7, 0x9d, 0x6e, 0x8f, 0x88, 0xbe, 0xdc, 0xa6, - 0x75, 0xd2, 0xef, 0x93, 0x8a, 0xb3, 0xeb, 0x07, 0xba, 0xfb, 0x2e, 0xb2, 0x4c, 0x0a, 0x34, 0xea, - 0x63, 0x1c, 0x23, 0x6a, 0x32, 0x12, 0x98, 0x56, 0xe0, 0x0e, 0x06, 0xa4, 0x57, 0xeb, 0x3f, 0x36, - 0xc0, 0x8c, 0x7f, 0xa9, 0x80, 0x68, 0x51, 0xb9, 0x23, 0x3e, 0x1e, 0x66, 0x5e, 0x8a, 0x84, 0x45, - 0x80, 0x0c, 0x22, 0x51, 0xad, 0xc0, 0xf1, 0x02, 0x01, 0xc9, 0x10, 0x25, 0x21, 0xd5, 0x0a, 0xc0, - 0x04, 0xa9, 0xe5, 0x71, 0xb7, 0xd7, 0xfb, 0x8a, 0x7b, 0x7c, 0xd0, 0x25, 0x4a, 0x73, 0x05, 0x16, - 0xca, 0x9d, 0x4e, 0x5c, 0x84, 0xcc, 0x49, 0x22, 0xe3, 0xac, 0xfa, 0x04, 0x6e, 0x8a, 0x6a, 0x1a, - 0x69, 0x27, 0x81, 0x9a, 0x26, 0x83, 0x22, 0x0d, 0x26, 0x30, 0xb9, 0xf5, 0x6d, 0xe5, 0x05, 0x37, - 0xe9, 0x48, 0x24, 0x48, 0xe6, 0x25, 0xba, 0xf7, 0x36, 0x45, 0xd1, 0x20, 0xc5, 0x4a, 0x58, 0xcc, - 0x50, 0x5d, 0xa1, 0x57, 0x41, 0x1c, 0x32, 0xb1, 0x59, 0xff, 0xd1, 0xcf, 0x56, 0x2f, 0xfd, 0xe0, - 0xe7, 0xab, 0xc6, 0x8f, 0x7e, 0xbe, 0x6a, 0xfc, 0xf4, 0xe7, 0xab, 0x97, 0xbe, 0xf7, 0xef, 0xab, - 0xc6, 0x57, 0xee, 0x49, 0xff, 0x43, 0xe6, 0xd8, 0x09, 0xbc, 0xee, 0x0b, 0x97, 0x3a, 0x16, 0xa2, - 0xd0, 0xc7, 0x77, 0x06, 0x47, 0x87, 0x77, 0x06, 0x07, 0x77, 0x22, 0x37, 0xe9, 0x60, 0x8a, 0x7e, - 0xe9, 0xed, 0xde, 0x7f, 0x07, 0x00, 0x00, 0xff, 0xff, 0x2d, 0x89, 0x43, 0xdc, 0x9f, 0x66, 0x00, - 0x00, + // 6266 bytes of a gzipped FileDescriptorProto + 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xec, 0x7d, 0xcd, 0x6f, 0x1c, 0xc9, + 0x75, 0xb8, 0x7a, 0x66, 0x48, 0x0e, 0xdf, 0x90, 0x54, 0xb3, 0x48, 0x4a, 0x23, 0x4a, 0xa2, 0xb8, + 0xb3, 0xf2, 0xae, 0x96, 0x5e, 0x53, 0xfe, 0x49, 0xde, 0xc5, 0xda, 0x3f, 0x59, 0xbb, 0xc3, 0x99, + 0x91, 0x38, 0xd2, 0x68, 0xc8, 0xed, 0x69, 0x6a, 0x6d, 0x03, 0x0b, 0xa2, 0x39, 0x53, 0xa2, 0xda, + 0x1c, 0x4e, 0x8f, 0xbb, 0x9b, 0x5a, 0xe9, 0xf7, 0x3b, 0x25, 0x88, 0x83, 0x20, 0x46, 0x82, 0x18, + 0x08, 0x12, 0x23, 0x89, 0xf3, 0x75, 0x0b, 0x72, 0xf0, 0xc5, 0xff, 0x41, 0x80, 0xc0, 0xa7, 0xc0, + 0x30, 0x90, 0x4b, 0x0e, 0x8e, 0xed, 0x5c, 0x02, 0xf8, 0x92, 0x53, 0x9c, 0x8b, 0x81, 0xa0, 0xbe, + 0xba, 0xab, 0xfa, 0x63, 0x7a, 0x86, 0xa4, 0x1d, 0x3b, 0xf0, 0x89, 0xd3, 0x55, 0xef, 0x55, 0x55, + 0x57, 0xbd, 0xef, 0xf7, 0xaa, 0x09, 0x7a, 0xdf, 0x39, 0xf4, 0xb0, 0xfb, 0xc2, 0xee, 0xe2, 0xcd, + 0xa1, 0xeb, 0xf8, 0x0e, 0x82, 0xb0, 0x65, 0xf5, 0x33, 0x87, 0xb6, 0xff, 0xfc, 0xe4, 0x60, 0xb3, + 0xeb, 0x1c, 0xdf, 0x3e, 0x74, 0x0e, 0x9d, 0xdb, 0x14, 0xe4, 0xe0, 0xe4, 0x19, 0x7d, 0xa2, 0x0f, + 0xf4, 0x17, 0x43, 0x5d, 0xbd, 0x71, 0xe8, 0x38, 0x87, 0x7d, 0x1c, 0x42, 0xf9, 0xf6, 0x31, 0xf6, + 0x7c, 0xeb, 0x78, 0xc8, 0x01, 0x16, 0x8e, 0xb1, 0x6f, 0xf5, 0x2c, 0xdf, 0xe2, 0xcf, 0x17, 0x23, + 0x00, 0x95, 0xff, 0x00, 0x98, 0xa9, 0xb5, 0x3b, 0xbe, 0xe3, 0x62, 0x84, 0xa0, 0xb0, 0xb7, 0xd7, + 0xac, 0x97, 0xb5, 0x75, 0xed, 0xd6, 0xac, 0x41, 0x7f, 0xa3, 0x37, 0x60, 0xa1, 0xc3, 0xd6, 0x56, + 0xed, 0xf5, 0x5c, 0xec, 0x79, 0xe5, 0x1c, 0xed, 0x8d, 0xb4, 0xa2, 0x35, 0x80, 0xce, 0x87, 0x2d, + 0x01, 0x93, 0xa7, 0x30, 0x52, 0x0b, 0xda, 0x04, 0xd4, 0x72, 0xba, 0x47, 0x91, 0xb1, 0x0a, 0x14, + 0x2e, 0xa1, 0x07, 0xdd, 0x84, 0x82, 0xe1, 0xf4, 0x71, 0x79, 0x7a, 0x5d, 0xbb, 0xb5, 0x70, 0x47, + 0xdf, 0x0c, 0xde, 0xa3, 0xd6, 0x26, 0xed, 0x06, 0xed, 0x25, 0x2b, 0x36, 0xed, 0xee, 0x51, 0x79, + 0x66, 0x5d, 0xbb, 0x55, 0x30, 0xe8, 0x6f, 0xf4, 0x69, 0x98, 0xea, 0xf8, 0x96, 0x8f, 0xcb, 0x45, + 0x8a, 0xba, 0xb2, 0x29, 0x6d, 0x78, 0xdb, 0xe9, 0x61, 0xda, 0x69, 0x30, 0x18, 0xf4, 0x45, 0x98, + 0x6e, 0x59, 0x07, 0xb8, 0xef, 0x95, 0x67, 0xd7, 0xf3, 0xb7, 0x4a, 0x77, 0x6e, 0xc8, 0xd0, 0x7c, + 0x5f, 0x36, 0x19, 0x44, 0x63, 0xe0, 0xbb, 0xaf, 0xb6, 0x0a, 0xdf, 0xfb, 0xe1, 0x8d, 0x0b, 0x06, + 0x47, 0x42, 0xff, 0x07, 0x66, 0x3f, 0x72, 0xdc, 0x23, 0x36, 0x1f, 0xd0, 0xf9, 0x96, 0xc2, 0xa5, + 0x06, 0x5d, 0x46, 0x08, 0x85, 0x2a, 0x30, 0xf7, 0xe1, 0x09, 0x76, 0x5f, 0x89, 0x2d, 0x28, 0xd1, + 0x2d, 0x50, 0xda, 0xd0, 0xbb, 0x00, 0x35, 0x67, 0xf0, 0xcc, 0x3e, 0xac, 0x5b, 0xbe, 0x55, 0x9e, + 0x5b, 0xd7, 0x6e, 0x95, 0xee, 0x5c, 0x52, 0x56, 0x16, 0xf4, 0x1a, 0x12, 0x24, 0x7a, 0x17, 0x8a, + 0x06, 0xf6, 0x9c, 0x13, 0xb7, 0x8b, 0xcb, 0xf3, 0x14, 0x6b, 0x59, 0xc6, 0x12, 0x7d, 0xfc, 0x25, + 0x02, 0x58, 0x74, 0x09, 0xa6, 0xf7, 0x86, 0xa6, 0x7d, 0x8c, 0xcb, 0x0b, 0xeb, 0xda, 0xad, 0xbc, + 0xc1, 0x9f, 0xd0, 0x67, 0x61, 0xa9, 0xf3, 0xdc, 0x72, 0x7b, 0x91, 0x53, 0xbb, 0x48, 0x97, 0x9c, + 0xd4, 0x85, 0x56, 0xa1, 0x58, 0x73, 0x8e, 0x8f, 0x6d, 0xbf, 0x59, 0x2f, 0xeb, 0x14, 0x2c, 0x78, + 0x46, 0x0f, 0x60, 0xed, 0xa9, 0x8d, 0x3f, 0x79, 0xc2, 0xb7, 0xa7, 0xda, 0x3b, 0xb6, 0x3d, 0xcf, + 0x76, 0x06, 0x9d, 0x93, 0xe1, 0xd0, 0x71, 0x7d, 0xdc, 0x2b, 0x2f, 0xae, 0x6b, 0xb7, 0x8a, 0x46, + 0x06, 0x14, 0xda, 0x86, 0x1b, 0x89, 0x10, 0x0f, 0xf1, 0x00, 0xbb, 0x96, 0x6f, 0x3b, 0x83, 0x32, + 0xa2, 0xf4, 0x90, 0x05, 0x86, 0xee, 0xc1, 0x15, 0x19, 0x64, 0xe7, 0x80, 0xec, 0x14, 0xee, 0x35, + 0x86, 0x4e, 0xf7, 0x79, 0x79, 0x89, 0x8e, 0x91, 0x0e, 0x80, 0xee, 0xc3, 0x6a, 0xe2, 0x04, 0x06, + 0xb6, 0x7a, 0xaf, 0xca, 0xcb, 0xf4, 0x5d, 0x46, 0x40, 0x90, 0xd9, 0xeb, 0xf5, 0xd6, 0x53, 0xdb, + 0xb3, 0x0f, 0xec, 0xbe, 0xed, 0xbf, 0xda, 0xb2, 0x5c, 0xd7, 0xc6, 0x2e, 0x43, 0x5f, 0xa1, 0xe8, + 0xe9, 0x00, 0xe8, 0x0b, 0x50, 0x96, 0xc7, 0x6e, 0x0e, 0x0e, 0xc9, 0x01, 0x30, 0xe4, 0x4b, 0x14, + 0x39, 0xb5, 0x1f, 0xd5, 0xe1, 0xba, 0x32, 0x70, 0x1d, 0x0f, 0xfb, 0xce, 0x2b, 0xdc, 0xdb, 0x25, + 0x22, 0xa1, 0xeb, 0xf4, 0xcb, 0x97, 0x29, 0x19, 0x8c, 0x06, 0x22, 0xe7, 0xa0, 0x00, 0x54, 0xbb, + 0xbe, 0xfd, 0x82, 0x6e, 0xec, 0xae, 0x8b, 0x87, 0x96, 0x8b, 0x7b, 0xe5, 0x32, 0x5d, 0x48, 0x16, + 0x58, 0x6c, 0x3d, 0x21, 0xc8, 0x03, 0x3c, 0xe8, 0xe2, 0x5e, 0xf9, 0x0a, 0x1d, 0x67, 0x34, 0x10, + 0xda, 0x85, 0x15, 0x05, 0xe0, 0x81, 0xeb, 0x0c, 0x7c, 0x1b, 0xbb, 0xe5, 0x55, 0xce, 0x0a, 0xa1, + 0xec, 0x33, 0xc5, 0x2f, 0xce, 0x0a, 0xc9, 0x88, 0xab, 0x6d, 0x28, 0x49, 0xbc, 0x8f, 0x74, 0xc8, + 0x1f, 0xe1, 0x57, 0x5c, 0x3c, 0x92, 0x9f, 0xe8, 0x2d, 0x98, 0x7a, 0x61, 0xf5, 0x4f, 0x30, 0x15, + 0x8a, 0x25, 0x99, 0xf7, 0x29, 0x5e, 0xcb, 0xf6, 0x7c, 0x83, 0x41, 0x7c, 0x21, 0xf7, 0x9e, 0xf6, + 0xa8, 0x50, 0x9c, 0xd2, 0xa7, 0x2b, 0x3f, 0xcb, 0xc3, 0x8c, 0x79, 0x0e, 0x22, 0x57, 0x08, 0xbf, + 0x7c, 0x92, 0xf0, 0x2b, 0x8c, 0x21, 0xfc, 0xde, 0x81, 0x69, 0xca, 0xc3, 0x5e, 0x79, 0x8a, 0x0a, + 0xbf, 0xcb, 0x32, 0xb4, 0xd9, 0xa6, 0x7d, 0xcd, 0xc1, 0x33, 0x47, 0x08, 0x3d, 0x06, 0x8c, 0xee, + 0xc0, 0x72, 0xcb, 0x39, 0xf4, 0x2d, 0xbb, 0x4f, 0x16, 0x84, 0x5d, 0xb1, 0xca, 0x69, 0xba, 0xca, + 0xc4, 0xbe, 0x14, 0xf1, 0x3f, 0x93, 0x2a, 0xfe, 0x55, 0x09, 0x38, 0x3b, 0xb6, 0x04, 0x8c, 0x4a, + 0x57, 0x48, 0x90, 0xae, 0x29, 0x52, 0xad, 0x94, 0x2e, 0xd5, 0x3e, 0x80, 0xab, 0xd5, 0x13, 0xdf, + 0x69, 0x0e, 0xba, 0x2e, 0x65, 0x7d, 0x4a, 0x70, 0xa1, 0xd8, 0x9a, 0xa3, 0xd4, 0x39, 0x0a, 0xe4, + 0x51, 0xa1, 0x58, 0xd4, 0x67, 0x2b, 0x7f, 0x90, 0x87, 0x62, 0xcb, 0x39, 0xfc, 0x15, 0x38, 0xfa, + 0x7b, 0x44, 0x53, 0x0c, 0xfb, 0x76, 0xd7, 0x12, 0x87, 0xbf, 0x2a, 0xc3, 0xb7, 0x9c, 0x43, 0xde, + 0x2d, 0x9d, 0x7f, 0x80, 0x11, 0x39, 0x9d, 0xe9, 0x49, 0xf4, 0x53, 0xcb, 0xe9, 0x5a, 0x84, 0xc7, + 0xe8, 0xd9, 0x47, 0xf4, 0x93, 0xe8, 0x13, 0xf3, 0x89, 0x67, 0xf4, 0x14, 0xde, 0x18, 0x29, 0x8a, + 0xc2, 0xa3, 0x28, 0xd2, 0xa3, 0x18, 0x13, 0xba, 0xf2, 0xcf, 0x05, 0x98, 0x23, 0xe7, 0x21, 0x08, + 0x1d, 0x95, 0x61, 0x86, 0x3d, 0xb0, 0x63, 0x29, 0x18, 0xe2, 0x11, 0x6d, 0x49, 0x1b, 0x96, 0xa3, + 0x1b, 0xf6, 0x46, 0x64, 0xc3, 0x82, 0x51, 0x36, 0x05, 0x20, 0x95, 0x1a, 0xd2, 0xb6, 0x2d, 0xc3, + 0x14, 0x53, 0x2d, 0xec, 0xd8, 0xd8, 0x03, 0x51, 0x99, 0x2d, 0x6c, 0xf5, 0xb0, 0xdb, 0xac, 0xd3, + 0xa3, 0x2b, 0x18, 0xc1, 0x33, 0x3d, 0x67, 0xec, 0x1e, 0x97, 0xa7, 0xf8, 0x39, 0x63, 0xf7, 0x18, + 0x7d, 0x0c, 0x8b, 0x6d, 0x67, 0xf0, 0xd4, 0xf1, 0xed, 0xc1, 0x61, 0xb0, 0xa4, 0x69, 0xba, 0xa4, + 0xdb, 0xa9, 0x4b, 0x8a, 0x61, 0xb0, 0xb5, 0xc5, 0x47, 0x42, 0x2e, 0x94, 0xf9, 0x6f, 0x4a, 0xa6, + 0xcd, 0x41, 0xd7, 0x72, 0x07, 0x54, 0xcc, 0x12, 0x7e, 0x25, 0xb3, 0xbc, 0x9b, 0xf5, 0xe2, 0x31, + 0x44, 0x36, 0x59, 0xea, 0xb8, 0xab, 0xff, 0x17, 0xe6, 0x95, 0x75, 0xc9, 0x92, 0xb6, 0xc0, 0x24, + 0xed, 0xb2, 0x2c, 0x69, 0x67, 0x25, 0xa1, 0xba, 0x5a, 0x87, 0x4b, 0xc9, 0x6f, 0x37, 0xd1, 0x28, + 0x8f, 0xe1, 0xfa, 0xc8, 0xd5, 0x4f, 0x32, 0x58, 0xe5, 0x5b, 0x1a, 0x2c, 0xa8, 0x2c, 0x84, 0x1e, + 0xa8, 0x94, 0x46, 0xc7, 0x29, 0xdd, 0x29, 0xa7, 0x6d, 0xe5, 0x56, 0x91, 0xb0, 0xc0, 0xf7, 0x7f, + 0x78, 0x43, 0x33, 0x54, 0x0a, 0xbd, 0x06, 0xb3, 0x62, 0xd8, 0x3a, 0x9d, 0xb8, 0x60, 0x84, 0x0d, + 0x68, 0x1d, 0x4a, 0x4d, 0x2f, 0xd8, 0x0d, 0x4a, 0x67, 0x45, 0x43, 0x6e, 0xaa, 0xfc, 0xbe, 0x16, + 0xda, 0x88, 0xd4, 0x5a, 0xdb, 0xdd, 0x33, 0x1d, 0xdf, 0xea, 0xf3, 0x17, 0x0b, 0x9e, 0x89, 0x24, + 0xad, 0xed, 0xee, 0x55, 0x5f, 0x58, 0x76, 0xdf, 0x3a, 0xe8, 0xb3, 0x97, 0xd4, 0x0c, 0xa5, 0x8d, + 0xe0, 0x3f, 0xc1, 0xc7, 0x0c, 0x9f, 0xd1, 0x74, 0xf0, 0x4c, 0xf0, 0x9f, 0xe0, 0xe3, 0x10, 0x9f, + 0x91, 0xb6, 0xd2, 0x56, 0xf9, 0x2d, 0x2d, 0x55, 0xf1, 0x9b, 0x96, 0x7b, 0x88, 0x7d, 0xf2, 0xba, + 0x5c, 0xf4, 0x05, 0x92, 0x32, 0x6c, 0x20, 0x4e, 0x87, 0x64, 0xf4, 0xb1, 0xdd, 0x90, 0x5a, 0x62, + 0xda, 0x20, 0x1f, 0xd7, 0x06, 0x95, 0x9f, 0xcf, 0x81, 0xce, 0x0d, 0xfd, 0x6d, 0x6c, 0xb9, 0xfe, + 0x01, 0xb6, 0xfc, 0x5f, 0x43, 0x4f, 0x68, 0x13, 0x90, 0x69, 0x79, 0x02, 0xb7, 0xe6, 0x62, 0x8b, + 0x88, 0xc3, 0x19, 0x4a, 0x00, 0x09, 0x3d, 0xb1, 0xad, 0x29, 0x26, 0x28, 0xca, 0x9b, 0x30, 0xdf, + 0x1c, 0xd8, 0x7e, 0xe8, 0xe1, 0xcc, 0x52, 0x20, 0xb5, 0x91, 0x40, 0x3d, 0x74, 0x3c, 0xcf, 0x1e, + 0xaa, 0x3a, 0x57, 0x6d, 0x24, 0xf3, 0xb1, 0x86, 0x47, 0x8e, 0x3d, 0xc0, 0x3d, 0xaa, 0x6d, 0x8b, + 0x86, 0xd2, 0xf6, 0x4b, 0x77, 0x7b, 0x52, 0x0c, 0x81, 0x85, 0xf1, 0xdc, 0x9b, 0x8b, 0x11, 0xf7, + 0xe6, 0xb3, 0xb0, 0x54, 0xed, 0x1e, 0xe1, 0x1e, 0x69, 0xb0, 0x06, 0xbd, 0x2d, 0xcb, 0xef, 0x3e, + 0xe7, 0x5e, 0x50, 0xc1, 0x48, 0xea, 0x22, 0x66, 0x05, 0x6f, 0xa9, 0xe3, 0xbe, 0xfd, 0x82, 0xec, + 0x7c, 0xf7, 0x28, 0xea, 0x0d, 0x8d, 0x02, 0x19, 0xc3, 0xa5, 0x42, 0xe7, 0xe5, 0x52, 0x2d, 0x9d, + 0x83, 0x4b, 0xb5, 0x9c, 0xe5, 0x52, 0x45, 0xde, 0xa7, 0x66, 0xf9, 0x56, 0xdf, 0x39, 0x64, 0xf6, + 0x3d, 0x1b, 0x62, 0x85, 0x0e, 0x91, 0x01, 0x85, 0xb6, 0xe0, 0x9a, 0x0c, 0x61, 0xe0, 0x67, 0x2e, + 0xf6, 0x9e, 0x87, 0xbb, 0xc2, 0x1c, 0xa4, 0x91, 0x30, 0xf1, 0x31, 0x5e, 0x58, 0x7d, 0xbb, 0x47, + 0x98, 0x87, 0xad, 0xe4, 0x32, 0x5d, 0xc9, 0x48, 0x98, 0x91, 0x4e, 0x5a, 0x39, 0xc3, 0x49, 0x1b, + 0xe9, 0x1e, 0x5e, 0xc9, 0x72, 0x0f, 0x33, 0x5d, 0xbc, 0xd5, 0x71, 0x5c, 0xbc, 0x57, 0x11, 0x17, + 0x8f, 0xbe, 0x15, 0xa3, 0x78, 0x26, 0xa0, 0xbd, 0xf2, 0x55, 0x6a, 0x13, 0xbc, 0x25, 0x33, 0xdc, + 0x48, 0x91, 0xce, 0xb9, 0x30, 0x6b, 0xdc, 0x71, 0xbc, 0xcb, 0x6b, 0xe7, 0xe4, 0x5d, 0x5e, 0x3f, + 0x93, 0x77, 0xb9, 0x76, 0x4a, 0xef, 0x92, 0x7b, 0x83, 0x26, 0xcc, 0xd5, 0xda, 0xd5, 0x7e, 0xdf, + 0xe9, 0x5a, 0x3e, 0xd1, 0x69, 0x71, 0x27, 0x73, 0x19, 0xa6, 0xa8, 0xc4, 0xe0, 0x0a, 0x8e, 0x3d, + 0x30, 0x43, 0xe0, 0x6b, 0x27, 0xd8, 0x23, 0xb2, 0x88, 0x69, 0x99, 0xb0, 0xa1, 0xf2, 0xf3, 0x02, + 0x2c, 0x0a, 0x4f, 0x63, 0xb4, 0x5a, 0x5b, 0x87, 0x92, 0x61, 0x3d, 0xf3, 0x55, 0x9d, 0x26, 0x37, + 0x25, 0x28, 0xbe, 0x7c, 0xa2, 0xe2, 0x8b, 0x29, 0x82, 0x42, 0x92, 0x22, 0x38, 0x9b, 0xe7, 0x91, + 0xac, 0xe6, 0xa6, 0x53, 0xd5, 0x9c, 0xaa, 0x52, 0x66, 0x4e, 0xe5, 0xa9, 0x14, 0x27, 0xf0, 0x54, + 0xbe, 0x00, 0xe5, 0x88, 0xbc, 0x0e, 0x85, 0xce, 0x2c, 0x63, 0xf8, 0xb4, 0xfe, 0x31, 0x84, 0x39, + 0x8c, 0x25, 0xcc, 0x37, 0x40, 0x8f, 0xda, 0xb0, 0xdc, 0xb9, 0x8d, 0xb5, 0x4f, 0xe0, 0x59, 0xcd, + 0x4d, 0xe4, 0x59, 0x35, 0xa0, 0x24, 0x05, 0x10, 0x46, 0xf8, 0x55, 0x23, 0xed, 0xd9, 0xca, 0xd7, + 0xa7, 0x40, 0x37, 0xcf, 0xd3, 0x38, 0x0b, 0x43, 0x1e, 0xf9, 0x49, 0x42, 0x1e, 0xc9, 0x64, 0x57, + 0x48, 0x25, 0xbb, 0xb4, 0x10, 0xc9, 0xd4, 0xc4, 0x21, 0x92, 0xe9, 0x31, 0x43, 0x24, 0xc5, 0x53, + 0x87, 0x48, 0x66, 0xc7, 0x0f, 0x91, 0x40, 0xba, 0x65, 0x44, 0xc4, 0x08, 0x1e, 0xf6, 0xad, 0x57, + 0xb8, 0xd7, 0xf2, 0x18, 0xbd, 0x15, 0x0c, 0xb9, 0xe9, 0xec, 0x41, 0x94, 0x34, 0x0b, 0x6b, 0xfe, + 0xd4, 0x16, 0xd6, 0x42, 0xa6, 0x85, 0xf5, 0xa8, 0x50, 0x9c, 0xd1, 0x8b, 0x95, 0xef, 0x4c, 0x41, + 0xd1, 0xe8, 0x3c, 0x61, 0x06, 0xaf, 0x0e, 0x79, 0xd3, 0x73, 0x84, 0x27, 0x68, 0x7a, 0x0e, 0x91, + 0xd0, 0xcd, 0x41, 0x0f, 0xbf, 0x14, 0x12, 0x9a, 0x3e, 0x10, 0x79, 0xd8, 0xc2, 0x96, 0x87, 0xb7, + 0x9d, 0x3e, 0xf3, 0xee, 0x99, 0x8b, 0xa4, 0x36, 0x92, 0xe3, 0x30, 0xdd, 0x93, 0x01, 0x91, 0xfe, + 0x74, 0xe7, 0xb8, 0x9f, 0x24, 0xb7, 0xa1, 0x47, 0x30, 0xc7, 0x90, 0x6c, 0xcf, 0x77, 0xdc, 0x57, + 0x5c, 0x6e, 0x2a, 0x01, 0x08, 0xb1, 0xba, 0x4d, 0x19, 0x90, 0xf9, 0xdd, 0x0a, 0x2e, 0x3b, 0xa8, + 0xaf, 0x9d, 0xd8, 0x2e, 0x9b, 0x6e, 0x5a, 0x1c, 0x54, 0xd0, 0x84, 0xde, 0x86, 0xc5, 0x8f, 0xaa, + 0x2d, 0x03, 0x77, 0x1d, 0xb2, 0x1b, 0x75, 0xfb, 0x10, 0x7b, 0x3e, 0x0f, 0xd5, 0xc5, 0x3b, 0xd0, + 0xe7, 0x60, 0x45, 0x6a, 0xa4, 0x33, 0xd6, 0x9c, 0x93, 0x81, 0x4f, 0x29, 0xb2, 0x60, 0x24, 0x77, + 0x92, 0x83, 0x91, 0x3a, 0x6a, 0xce, 0xf1, 0xb0, 0x8f, 0x89, 0xd5, 0x34, 0xf0, 0x5d, 0x1b, 0x33, + 0x9a, 0x2c, 0x18, 0xa3, 0x40, 0x08, 0xbb, 0x48, 0xdd, 0x5b, 0x96, 0x87, 0xc9, 0xeb, 0x00, 0x45, + 0x4c, 0xe8, 0x89, 0xc0, 0xb7, 0x2c, 0xcf, 0x0f, 0xe9, 0x34, 0xa1, 0x07, 0x3d, 0x82, 0x75, 0xa9, + 0x75, 0xc7, 0xb5, 0x0f, 0xed, 0x81, 0xd5, 0x57, 0x0f, 0x74, 0x8e, 0x62, 0x67, 0xc2, 0x11, 0xc2, + 0x4d, 0x78, 0x15, 0x4a, 0xb8, 0x45, 0x23, 0xa9, 0x6b, 0xf5, 0x7d, 0x58, 0x8c, 0x1d, 0x64, 0x56, + 0x08, 0xa2, 0x20, 0x87, 0x20, 0x3e, 0x86, 0x59, 0xaa, 0x4a, 0xbb, 0x8e, 0xdb, 0x23, 0x88, 0xe4, + 0x65, 0x39, 0x22, 0x79, 0xbb, 0x0d, 0x28, 0x98, 0xaf, 0x86, 0x0c, 0x6f, 0x41, 0x15, 0x1b, 0x0c, + 0x87, 0xf4, 0x1a, 0x14, 0x86, 0xc8, 0x5b, 0x2a, 0x62, 0x08, 0xf9, 0xce, 0x19, 0xf4, 0x77, 0xe5, + 0x47, 0x39, 0x00, 0x3a, 0x3e, 0x35, 0x38, 0x08, 0x48, 0xdb, 0x3a, 0xc6, 0x42, 0x24, 0x93, 0xdf, + 0xb2, 0xcc, 0xcf, 0xa9, 0x32, 0x9f, 0x2f, 0x27, 0x1f, 0x2e, 0xa7, 0x0c, 0x33, 0x4f, 0xac, 0x97, + 0x1d, 0xfb, 0xff, 0x89, 0x38, 0x81, 0x78, 0x24, 0xfa, 0x41, 0x88, 0xe5, 0x3a, 0x0f, 0x83, 0x85, + 0x0d, 0x34, 0x3e, 0xd6, 0x6e, 0xd6, 0x39, 0x15, 0xd3, 0xdf, 0xe8, 0x73, 0x90, 0x33, 0x3b, 0x5c, + 0xd5, 0xaf, 0x6e, 0xb2, 0x04, 0xe9, 0xa6, 0x48, 0x90, 0x4a, 0xb6, 0x19, 0x8d, 0xb0, 0xfc, 0xd1, + 0xbf, 0xde, 0xd0, 0x8c, 0x9c, 0xd9, 0x21, 0xca, 0xb7, 0x39, 0xe8, 0xf6, 0x4f, 0x7a, 0xb8, 0xf1, + 0x72, 0x48, 0x38, 0x81, 0x2b, 0x21, 0x2e, 0xdf, 0xb0, 0xc7, 0x43, 0x8b, 0x19, 0x50, 0xc4, 0x6c, + 0x6d, 0xbc, 0xa4, 0x10, 0xdb, 0x96, 0xdb, 0xab, 0x3b, 0x9f, 0x0c, 0x62, 0x03, 0x31, 0x3b, 0x20, + 0x0b, 0xac, 0x52, 0x01, 0x30, 0x3d, 0x47, 0xec, 0xf0, 0x32, 0x4c, 0x31, 0xb6, 0x62, 0x87, 0xc8, + 0x1e, 0x2a, 0x3f, 0xd5, 0x88, 0xf5, 0x48, 0xf5, 0x23, 0x4d, 0x38, 0x24, 0xea, 0xc6, 0xbb, 0x30, + 0xbb, 0x33, 0x94, 0x83, 0x24, 0x91, 0xe0, 0x70, 0xad, 0x4d, 0x71, 0x77, 0x86, 0x46, 0x08, 0x87, + 0xb6, 0x82, 0xc4, 0x28, 0x53, 0x94, 0x37, 0x13, 0x12, 0xa3, 0x14, 0x20, 0x3d, 0x3b, 0x7a, 0xde, + 0xe9, 0x93, 0x4a, 0x0b, 0x4a, 0xb5, 0x76, 0x18, 0x78, 0x48, 0x7a, 0xd7, 0xb7, 0x44, 0x10, 0x3c, + 0x97, 0x9e, 0x8c, 0x65, 0x10, 0x95, 0x1f, 0xf3, 0xbd, 0xb3, 0xfc, 0x11, 0x7b, 0x37, 0xfe, 0x78, + 0xd9, 0x3b, 0x26, 0x26, 0xfa, 0x25, 0xee, 0xd8, 0x37, 0x8b, 0x30, 0x23, 0x28, 0x48, 0x71, 0x18, + 0x34, 0x61, 0x69, 0xf1, 0x06, 0xb4, 0x09, 0xd3, 0x4f, 0xb0, 0xff, 0xdc, 0xe9, 0x25, 0x89, 0x04, + 0xd6, 0x43, 0x45, 0x02, 0x87, 0x42, 0xf7, 0x64, 0xfe, 0xa7, 0xac, 0x1c, 0xb1, 0x3e, 0xc2, 0x5e, + 0xfe, 0x8e, 0xb2, 0xbc, 0xa8, 0xd2, 0x68, 0x68, 0x60, 0xd2, 0x51, 0xa6, 0x2f, 0xdd, 0xb9, 0x1e, + 0x8d, 0x86, 0x2a, 0x76, 0x9f, 0xa1, 0xa0, 0xa0, 0xfb, 0x84, 0x18, 0xc2, 0x11, 0xa6, 0xe8, 0x08, + 0xd7, 0x12, 0xa8, 0x34, 0x1c, 0x40, 0x46, 0x20, 0xf8, 0xa6, 0x84, 0x3f, 0x1d, 0xc7, 0x37, 0x63, + 0xf8, 0x12, 0x02, 0x31, 0xbf, 0x42, 0xf6, 0x4c, 0xf2, 0x2c, 0xc2, 0x5e, 0x43, 0x66, 0xe4, 0x7b, + 0xaa, 0xbf, 0xc7, 0x0d, 0xb7, 0xb2, 0xba, 0xf0, 0xb0, 0xdf, 0x50, 0xbd, 0xc3, 0x7b, 0x2a, 0xbf, + 0xf3, 0xcc, 0x58, 0x39, 0x8d, 0x39, 0x0d, 0x55, 0x3a, 0x7c, 0x5e, 0x61, 0x20, 0xaa, 0x2c, 0x23, + 0x26, 0xb0, 0xd4, 0x6d, 0x28, 0xcc, 0x76, 0x4f, 0x65, 0x16, 0xaa, 0x38, 0x13, 0x26, 0x16, 0xfd, + 0x86, 0xca, 0x5a, 0xef, 0xc3, 0x7c, 0x1d, 0x13, 0xc5, 0xc6, 0x97, 0xc3, 0x83, 0x7b, 0x57, 0x94, + 0xa8, 0x81, 0x0c, 0x60, 0xa8, 0xf0, 0x68, 0x0b, 0x16, 0x76, 0x5d, 0xe7, 0xe5, 0xab, 0xf0, 0xc0, + 0xe6, 0xb9, 0x80, 0x97, 0x46, 0x50, 0x21, 0x8c, 0x08, 0x06, 0xd1, 0xc2, 0xd1, 0x44, 0x41, 0xfb, + 0xe4, 0x98, 0x1a, 0x81, 0x05, 0x23, 0xa9, 0x0b, 0x6d, 0x49, 0xa9, 0x96, 0xc0, 0x1d, 0xbc, 0x98, + 0xee, 0x0e, 0x1a, 0x71, 0x70, 0xba, 0xe7, 0xcf, 0x71, 0xf7, 0x68, 0x1b, 0x5b, 0x7d, 0xff, 0x39, + 0x0d, 0x07, 0x46, 0xf7, 0x3c, 0xec, 0x36, 0x64, 0x58, 0x64, 0xc2, 0x72, 0xa7, 0xfb, 0x1c, 0xf7, + 0x4e, 0xfa, 0x98, 0x9b, 0xa8, 0xd4, 0x48, 0xa7, 0x81, 0xc1, 0xd2, 0x9d, 0x75, 0x79, 0x8c, 0x24, + 0x38, 0x23, 0x11, 0xbb, 0xe2, 0x40, 0x89, 0x72, 0xa2, 0x37, 0x74, 0x06, 0x1e, 0x1e, 0xe1, 0x9a, + 0x71, 0x35, 0x9d, 0x53, 0xd4, 0xb4, 0x30, 0x9c, 0x98, 0xf2, 0x16, 0x8f, 0xa3, 0x92, 0x58, 0x95, + 0x4d, 0x40, 0x12, 0x3d, 0x4b, 0xf3, 0x3e, 0xb0, 0x5d, 0x49, 0x18, 0x89, 0xc7, 0xca, 0x7f, 0x16, + 0x68, 0x3c, 0x97, 0x81, 0x9d, 0xaf, 0xd4, 0xba, 0x06, 0xb3, 0x0d, 0xd7, 0x75, 0xdc, 0x9a, 0xd3, + 0xc3, 0xf4, 0x15, 0xe6, 0x8d, 0xb0, 0x81, 0x98, 0xe2, 0xf4, 0xe1, 0x09, 0xf6, 0x3c, 0xeb, 0x10, + 0xf3, 0xf8, 0x85, 0xd2, 0x86, 0xd6, 0x00, 0x9a, 0xde, 0x76, 0xf5, 0x31, 0xc6, 0x43, 0xec, 0x52, + 0xa9, 0x53, 0x34, 0xa4, 0x16, 0xf4, 0xbe, 0xb2, 0xbb, 0x5c, 0xac, 0x5c, 0x8e, 0x09, 0x46, 0xd6, + 0xcd, 0x25, 0xa3, 0x72, 0x1e, 0x84, 0xd1, 0x24, 0x27, 0x86, 0x4b, 0x16, 0x95, 0xd1, 0xa4, 0x7e, + 0x43, 0x81, 0x26, 0xd4, 0x46, 0x65, 0x0d, 0x9f, 0xbe, 0x18, 0x9f, 0x5e, 0xea, 0x36, 0x64, 0x58, + 0xc2, 0x62, 0xb5, 0xfe, 0x89, 0xe7, 0x63, 0xb7, 0x8e, 0x89, 0x77, 0xea, 0x71, 0xe1, 0xa2, 0xb0, + 0x98, 0x0a, 0x61, 0x44, 0x30, 0xd0, 0x7d, 0x98, 0x0d, 0x53, 0x5c, 0x90, 0x40, 0xa6, 0xa2, 0x93, + 0x11, 0x28, 0xf6, 0x4e, 0xfa, 0xbe, 0x11, 0xa2, 0xa0, 0xfb, 0x00, 0x92, 0x68, 0x64, 0x32, 0x66, + 0x4d, 0x1e, 0x20, 0x4e, 0x48, 0x06, 0x44, 0xc4, 0x23, 0x61, 0x20, 0xec, 0x32, 0x09, 0x37, 0x97, + 0xb0, 0x79, 0x52, 0xbf, 0xa1, 0x40, 0x57, 0x1e, 0xd1, 0x98, 0x19, 0xb3, 0x7f, 0x83, 0x6d, 0x79, + 0x87, 0x68, 0x50, 0xd2, 0xe2, 0x95, 0x35, 0xaa, 0xd7, 0x57, 0x62, 0x87, 0x49, 0x7a, 0xf9, 0x51, + 0x0a, 0xd8, 0xca, 0xeb, 0xca, 0x41, 0x10, 0xf3, 0xed, 0x29, 0xd5, 0xdb, 0xdc, 0x7c, 0xa3, 0x0f, + 0x95, 0x87, 0x30, 0x6f, 0x5a, 0xde, 0x91, 0x69, 0x1d, 0xf4, 0xf1, 0x9e, 0x87, 0x5d, 0xc2, 0x46, + 0xe4, 0xef, 0x20, 0xb4, 0xa5, 0x83, 0x67, 0xd2, 0xb7, 0x6b, 0x79, 0xde, 0x27, 0x8e, 0xdb, 0xe3, + 0xc1, 0x8d, 0xe0, 0xb9, 0xf2, 0x0d, 0x8d, 0xac, 0x92, 0xca, 0xad, 0x44, 0x33, 0x26, 0xdd, 0x16, + 0x57, 0xe2, 0x2f, 0xf9, 0x68, 0x3e, 0x31, 0xc8, 0x58, 0x17, 0xe4, 0x8c, 0xf5, 0x1a, 0xd5, 0xfd, + 0xaa, 0x51, 0x2e, 0xb5, 0x54, 0xfe, 0x2c, 0x47, 0x68, 0x78, 0xf0, 0xcc, 0x3e, 0xac, 0x3d, 0xb7, + 0x06, 0x87, 0x18, 0xdd, 0x0d, 0x56, 0xc7, 0xf3, 0x9e, 0x4b, 0xaa, 0xc3, 0x41, 0xbb, 0xc2, 0x1d, + 0x64, 0xef, 0x71, 0x0f, 0x80, 0xa1, 0x4b, 0x8e, 0xca, 0xb5, 0x78, 0x7c, 0x23, 0x84, 0x31, 0x24, + 0x78, 0x64, 0xc2, 0x42, 0x73, 0x60, 0xfb, 0xb6, 0xd5, 0x7f, 0x82, 0x8f, 0x0f, 0xb0, 0x2b, 0xac, + 0xb2, 0xb7, 0xd3, 0x46, 0xd8, 0x54, 0xc1, 0x99, 0xeb, 0x1c, 0x19, 0x63, 0xb5, 0x0a, 0x4b, 0x09, + 0x60, 0x13, 0xe5, 0x86, 0xdf, 0x82, 0xf9, 0xce, 0xf3, 0x13, 0xbf, 0xe7, 0x7c, 0x32, 0x60, 0xaa, + 0x8d, 0x9c, 0x0d, 0x0d, 0xcb, 0x89, 0x23, 0x13, 0x8f, 0x95, 0xbf, 0x9b, 0x82, 0x8b, 0x11, 0x11, + 0x9e, 0x78, 0xba, 0x37, 0x61, 0x7e, 0xcb, 0x71, 0x7c, 0xcf, 0x77, 0xad, 0xe1, 0xd0, 0x1e, 0x1c, + 0xd2, 0x49, 0x8b, 0x86, 0xda, 0x48, 0x44, 0x03, 0x8f, 0xd9, 0xd0, 0x0d, 0xcd, 0xd3, 0x0d, 0x55, + 0x44, 0x83, 0xd4, 0x6d, 0xc8, 0xb0, 0x4c, 0x26, 0x85, 0x5b, 0xc5, 0xcd, 0xb5, 0x72, 0xda, 0x56, + 0x1a, 0xea, 0xe9, 0xbf, 0x1f, 0x79, 0x63, 0x6e, 0xab, 0x5d, 0x51, 0x05, 0x83, 0x04, 0x60, 0x44, + 0x76, 0xe8, 0x31, 0x2c, 0xb2, 0xc0, 0x9a, 0x14, 0x69, 0xe3, 0x92, 0x55, 0x31, 0x19, 0x63, 0x40, + 0x46, 0x1c, 0x2f, 0x6e, 0x8a, 0xcc, 0x4c, 0x68, 0x8a, 0x3c, 0x86, 0xc5, 0x47, 0x8e, 0x3d, 0x60, + 0x51, 0x6d, 0x2e, 0xff, 0xb8, 0xa0, 0x55, 0x56, 0x13, 0x03, 0x32, 0xe2, 0x78, 0x68, 0x1b, 0x74, + 0x36, 0x3a, 0xb5, 0x55, 0xd8, 0x82, 0x66, 0xe3, 0xa6, 0x68, 0x14, 0xc6, 0x88, 0x61, 0x91, 0xe3, + 0xad, 0xf6, 0x7a, 0x82, 0x0b, 0x93, 0x6c, 0x3b, 0xa9, 0xdb, 0x90, 0x61, 0x89, 0xe4, 0x0f, 0x48, + 0x85, 0x61, 0x97, 0xe2, 0x92, 0x5f, 0x85, 0x30, 0x22, 0x18, 0x15, 0x9c, 0x6c, 0xab, 0x24, 0xd2, + 0x6b, 0x84, 0x12, 0x73, 0xe3, 0x53, 0x62, 0xe5, 0x4f, 0x72, 0x70, 0x29, 0x12, 0xae, 0x13, 0x09, + 0xa3, 0x4a, 0x60, 0x1a, 0x93, 0x49, 0x98, 0xb4, 0x9e, 0x35, 0x94, 0x36, 0x1a, 0x6c, 0x93, 0x61, + 0x72, 0x0c, 0x46, 0x6e, 0x23, 0x72, 0xb6, 0xf1, 0x92, 0x88, 0x20, 0xdb, 0xe7, 0x05, 0x14, 0xc1, + 0x33, 0x31, 0x21, 0xf9, 0x78, 0xa6, 0x7d, 0x8c, 0x9d, 0x13, 0xdf, 0xb4, 0xbb, 0x47, 0x1e, 0x97, + 0x8e, 0x49, 0x5d, 0x04, 0xc3, 0x4c, 0xc0, 0x60, 0x42, 0x33, 0xa9, 0x0b, 0x7d, 0x0e, 0x56, 0x1a, + 0x44, 0x5c, 0x58, 0x3e, 0xae, 0x9d, 0xb8, 0x2e, 0x1e, 0xf8, 0x14, 0xc6, 0xe3, 0x59, 0x8e, 0xe4, + 0xce, 0xca, 0xed, 0x04, 0xaa, 0x64, 0xaf, 0x62, 0x7b, 0xb4, 0x16, 0x84, 0x6d, 0x47, 0xf0, 0x5c, + 0xe9, 0x27, 0x30, 0x15, 0xba, 0x0b, 0x05, 0xa2, 0x6f, 0xb8, 0x94, 0x56, 0x78, 0x42, 0x51, 0x54, + 0x5c, 0x56, 0x53, 0x60, 0xba, 0xa9, 0x96, 0x77, 0x54, 0xb7, 0x7c, 0xeb, 0xc0, 0xf2, 0x84, 0xc8, + 0x53, 0xda, 0x88, 0xd4, 0x53, 0xb9, 0x28, 0x5d, 0xea, 0xbd, 0x1d, 0x67, 0x89, 0x11, 0xd0, 0x6f, + 0x2a, 0x64, 0x9f, 0x6e, 0xcd, 0x56, 0x7e, 0xa6, 0x45, 0xa9, 0xfc, 0xb4, 0x59, 0x09, 0xf4, 0x34, + 0x45, 0xb7, 0x6c, 0xa6, 0xf3, 0xcb, 0x38, 0xda, 0x85, 0xf0, 0x0a, 0x39, 0x43, 0x9e, 0x57, 0xa0, + 0xbf, 0xcf, 0x43, 0xe3, 0xfc, 0x7d, 0x5e, 0x35, 0x29, 0x83, 0xaa, 0x32, 0x4d, 0xaa, 0x2a, 0xfb, + 0x22, 0xab, 0x6c, 0xb0, 0x06, 0x3d, 0x51, 0xdf, 0x76, 0x75, 0x84, 0x7f, 0x21, 0xf2, 0x5e, 0x02, + 0x85, 0x6c, 0xa5, 0x08, 0xc7, 0x73, 0xcf, 0x40, 0x84, 0xe0, 0x6b, 0x00, 0x1c, 0x8a, 0x30, 0x5c, + 0x81, 0x0e, 0x7d, 0x7d, 0xc4, 0xd0, 0xcd, 0xba, 0x88, 0x17, 0x84, 0x68, 0xe8, 0x23, 0x58, 0x49, + 0x4c, 0x7a, 0x71, 0x55, 0xf2, 0x9a, 0x3c, 0x5e, 0x72, 0xc5, 0x75, 0x32, 0x7e, 0x76, 0x9a, 0x7c, + 0x7a, 0x9c, 0x34, 0xf9, 0xa3, 0xb4, 0xdc, 0xf0, 0x4c, 0x7a, 0x6e, 0x38, 0x25, 0x2b, 0x5c, 0xf9, + 0xf3, 0x5c, 0xca, 0xbb, 0x12, 0xa2, 0x64, 0x39, 0x6d, 0xc6, 0xce, 0x84, 0x46, 0xc2, 0x06, 0x72, + 0x02, 0x8d, 0x01, 0xe1, 0xcf, 0x1e, 0x57, 0xff, 0xe2, 0x31, 0xa5, 0xec, 0xf0, 0x0e, 0x2c, 0x07, + 0xe5, 0x0a, 0xb4, 0x24, 0x9d, 0x25, 0x00, 0x38, 0xf1, 0x25, 0xf6, 0xa1, 0x4d, 0x40, 0x09, 0x25, + 0x19, 0x4c, 0x96, 0x25, 0xf4, 0x44, 0xea, 0xb3, 0xa6, 0x63, 0xf5, 0x59, 0xcb, 0x30, 0xc5, 0xca, + 0x19, 0x58, 0x9d, 0x12, 0x7b, 0x20, 0x52, 0x8b, 0xbc, 0xb4, 0x1f, 0xd6, 0x73, 0x06, 0xcf, 0x95, + 0x7f, 0xd1, 0xd4, 0xaa, 0x8c, 0x60, 0x77, 0x84, 0x16, 0x90, 0xa5, 0xb7, 0x36, 0x9e, 0xf4, 0xce, + 0xa5, 0x4b, 0xef, 0x77, 0xe1, 0x52, 0x28, 0x85, 0x14, 0x24, 0xb6, 0x97, 0x29, 0xbd, 0xe9, 0x32, + 0xbc, 0x30, 0x4a, 0x86, 0xff, 0xc3, 0x3c, 0x94, 0xf8, 0x2a, 0xa8, 0x37, 0x24, 0xaa, 0x7c, 0x35, + 0xa9, 0xca, 0xf7, 0x7f, 0x57, 0x15, 0x5a, 0x35, 0x88, 0x99, 0x16, 0xa9, 0x60, 0x78, 0x3d, 0x21, + 0x90, 0x45, 0x2b, 0x4b, 0xc7, 0xbc, 0x82, 0x33, 0x7b, 0xaa, 0x2b, 0x38, 0x90, 0x5c, 0xfb, 0xa6, + 0x16, 0x33, 0x94, 0xc6, 0xa9, 0x6a, 0x9b, 0xcb, 0xac, 0x6a, 0x9b, 0x3f, 0x55, 0x55, 0xdb, 0xc2, + 0xa9, 0x2e, 0xf3, 0x5c, 0x1c, 0xe7, 0x32, 0x8f, 0x3e, 0x5e, 0xb5, 0xdb, 0x62, 0xa4, 0xda, 0x2d, + 0x23, 0xb3, 0x8a, 0xce, 0xa3, 0x76, 0x6d, 0xe9, 0xbc, 0x6a, 0xd7, 0x96, 0xcf, 0xa1, 0x76, 0x6d, + 0xe5, 0xec, 0xb5, 0x6b, 0x97, 0xce, 0xa5, 0x76, 0xed, 0xf2, 0x39, 0xd4, 0xae, 0x95, 0xc7, 0xa8, + 0x5d, 0x1b, 0x7d, 0xbd, 0xe9, 0x4a, 0xe6, 0xf5, 0xa6, 0x51, 0xb5, 0x6f, 0xab, 0x67, 0xa9, 0x7d, + 0xbb, 0x7a, 0xe6, 0xda, 0xb7, 0x6b, 0xe7, 0x74, 0xbd, 0xe9, 0xfa, 0x39, 0x15, 0xa0, 0xad, 0x9d, + 0xa9, 0x00, 0xed, 0xc6, 0xaf, 0xd6, 0xf5, 0xa6, 0xef, 0xe6, 0xd8, 0x8d, 0x52, 0x7e, 0xbd, 0x92, + 0x2b, 0x3e, 0x2d, 0xf9, 0x7a, 0xa5, 0xe5, 0xe3, 0x4d, 0x06, 0xa1, 0xc8, 0x76, 0xd6, 0x94, 0x7d, + 0x90, 0xb9, 0x71, 0x0e, 0x32, 0x75, 0xe3, 0xf2, 0xa7, 0xdd, 0x38, 0x03, 0x4a, 0xd2, 0xa2, 0x13, + 0x36, 0xee, 0x33, 0xea, 0xc6, 0x5d, 0x4e, 0x51, 0x6b, 0xb2, 0x95, 0xfe, 0x4f, 0x05, 0x5a, 0x32, + 0x75, 0x2e, 0xca, 0xff, 0x37, 0x55, 0x4e, 0xbf, 0xc6, 0x55, 0x4e, 0x19, 0x9a, 0x75, 0x7e, 0xdc, + 0x9a, 0xa5, 0xbf, 0xd0, 0xd8, 0x35, 0xc3, 0x4c, 0x3e, 0x34, 0xb3, 0xf8, 0xf0, 0x6c, 0xf4, 0x6e, + 0x26, 0xd3, 0xfb, 0xdf, 0xe6, 0x01, 0x24, 0x0f, 0x3f, 0x29, 0x4e, 0x24, 0x58, 0x20, 0x27, 0xb1, + 0xc0, 0x4d, 0x98, 0x27, 0x62, 0x07, 0x0f, 0x54, 0xd3, 0x56, 0x6d, 0x8c, 0x50, 0x4d, 0x61, 0x6c, + 0xaa, 0xc9, 0xb6, 0x49, 0xa6, 0xce, 0xcb, 0x26, 0x99, 0x3e, 0x07, 0x9b, 0x64, 0xe6, 0x6c, 0x57, + 0x94, 0x8b, 0x59, 0x3a, 0xbc, 0xf2, 0x37, 0x5a, 0x70, 0x48, 0x84, 0x8c, 0x3e, 0x88, 0x90, 0x51, + 0x25, 0x96, 0x7d, 0xcd, 0xa2, 0xa4, 0x0f, 0xb3, 0x28, 0xe9, 0x6d, 0x95, 0x92, 0x2e, 0x25, 0xcc, + 0xe0, 0xb8, 0x58, 0x26, 0xa4, 0x1f, 0xe4, 0xa2, 0xb9, 0xe1, 0xb4, 0x20, 0xb9, 0x4a, 0x38, 0xb9, + 0x6c, 0xc2, 0xc9, 0x9f, 0x23, 0xe1, 0x14, 0xce, 0x8b, 0x70, 0xa6, 0xce, 0x81, 0x70, 0xa6, 0x33, + 0x08, 0xa7, 0xf2, 0x9d, 0x42, 0x34, 0x1b, 0x88, 0xde, 0x81, 0x22, 0x67, 0x65, 0x71, 0xfc, 0x4b, + 0x09, 0x6c, 0x2e, 0xdc, 0x11, 0x01, 0x4a, 0xd0, 0x6a, 0x02, 0x2d, 0x17, 0x47, 0xab, 0xa9, 0x68, + 0x02, 0x14, 0xbd, 0x47, 0xeb, 0xd7, 0x38, 0x1e, 0xd3, 0x62, 0xcb, 0x49, 0xe5, 0x21, 0x1c, 0x31, + 0x04, 0x46, 0xf7, 0xa1, 0x14, 0x12, 0x8a, 0x88, 0x38, 0xa5, 0xd0, 0x91, 0x48, 0xc0, 0x4a, 0x08, + 0xa8, 0x2e, 0x42, 0x95, 0x3d, 0x3e, 0x02, 0xab, 0xb6, 0x2c, 0xc7, 0xe3, 0xf1, 0x3d, 0x79, 0x0c, + 0x15, 0x29, 0x3d, 0x62, 0x35, 0xfd, 0x8b, 0x8e, 0x58, 0xcd, 0x9c, 0xc9, 0x26, 0x2a, 0x9e, 0xd2, + 0x26, 0xaa, 0xfc, 0xae, 0x06, 0x25, 0x4e, 0x31, 0xd4, 0x7e, 0xf9, 0x3c, 0x25, 0x17, 0x66, 0x85, + 0x68, 0xdc, 0x0a, 0x09, 0xcc, 0x47, 0xde, 0xa3, 0xa4, 0x4e, 0x03, 0x70, 0x74, 0x8f, 0x9d, 0x3d, + 0xc3, 0xcd, 0xf1, 0xdd, 0x0f, 0x4d, 0x4f, 0x91, 0xc3, 0x90, 0x91, 0x43, 0x84, 0xca, 0xb7, 0x0b, + 0xb0, 0xc2, 0x43, 0xa6, 0x22, 0xf1, 0xc2, 0x4b, 0x6f, 0xde, 0x80, 0x85, 0xf6, 0xc9, 0xf1, 0xce, + 0xb3, 0x70, 0x70, 0x66, 0x5c, 0x45, 0x5a, 0x89, 0xa8, 0xa0, 0x2d, 0xc1, 0xfa, 0x99, 0x02, 0x52, + 0x1b, 0xd1, 0x06, 0xe8, 0x02, 0x2f, 0xb8, 0xd0, 0xc0, 0xa2, 0x42, 0xb1, 0x76, 0xe2, 0x93, 0xb7, + 0xf1, 0x4b, 0x3f, 0x28, 0x8e, 0xe0, 0x4f, 0xc8, 0x84, 0x12, 0xfb, 0xb5, 0xf5, 0xea, 0x31, 0x16, + 0x75, 0xbd, 0x77, 0x64, 0xda, 0x48, 0x7c, 0x93, 0x4d, 0x09, 0x89, 0x85, 0x92, 0xe5, 0x61, 0xd0, + 0xb3, 0xa4, 0xb2, 0x15, 0x76, 0x43, 0xf8, 0xbd, 0x31, 0xc6, 0x8e, 0xa2, 0x46, 0xaf, 0x0a, 0x07, + 0xa5, 0x2d, 0xd4, 0x96, 0x3b, 0x14, 0xb9, 0x36, 0x5e, 0xc2, 0x2a, 0xa2, 0x3d, 0xf1, 0x9e, 0xd5, + 0xfb, 0xa0, 0x47, 0x17, 0x9e, 0x7c, 0xdd, 0x25, 0xb9, 0xa6, 0x55, 0xb9, 0xe9, 0xab, 0x2c, 0x2e, + 0x6b, 0x14, 0x25, 0x1c, 0xfe, 0x73, 0x0d, 0xae, 0x18, 0xd8, 0x63, 0xf9, 0x83, 0x8f, 0x2c, 0x1f, + 0xbb, 0xc7, 0x96, 0x7b, 0x24, 0x68, 0x24, 0x3c, 0x29, 0x4d, 0x39, 0xa9, 0x2f, 0xa9, 0x27, 0x95, + 0x8b, 0xdf, 0x84, 0x4e, 0x1d, 0x33, 0xe3, 0xb4, 0x92, 0x77, 0x31, 0xff, 0x8b, 0xda, 0xc5, 0xca, + 0x7f, 0xf1, 0x4b, 0xef, 0x23, 0x3d, 0x8d, 0xdf, 0xdc, 0x0a, 0xfa, 0xcd, 0xad, 0xa0, 0xf3, 0xbf, + 0x15, 0xf4, 0xdd, 0x1c, 0xff, 0xfe, 0x05, 0x31, 0x26, 0xef, 0x07, 0x4e, 0x2a, 0x53, 0x0f, 0xeb, + 0x31, 0xf5, 0x4e, 0x4d, 0x49, 0x0a, 0xa2, 0x9a, 0x92, 0x4c, 0xfe, 0xde, 0x0f, 0x8c, 0xd1, 0xdc, + 0x28, 0xfc, 0x54, 0x53, 0xb4, 0x03, 0x25, 0x69, 0xf0, 0x84, 0x8c, 0xda, 0xa6, 0x6a, 0x8a, 0xa6, + 0xde, 0xd5, 0x97, 0x45, 0x54, 0x27, 0xcb, 0xbe, 0xcd, 0x1a, 0x34, 0xc9, 0x55, 0xfa, 0xf7, 0xa2, + 0x5a, 0xd6, 0x94, 0xc8, 0xb1, 0xef, 0x2b, 0xea, 0x37, 0x31, 0xf0, 0x10, 0x76, 0x0b, 0xbb, 0x47, + 0x56, 0xd8, 0x77, 0x03, 0x77, 0x91, 0xdb, 0xbd, 0x4b, 0x09, 0x4e, 0xa2, 0x28, 0xd2, 0x11, 0x8e, + 0xe5, 0xbb, 0xe1, 0x81, 0x72, 0x37, 0x6b, 0x39, 0xe9, 0x18, 0x42, 0x8e, 0xe0, 0x87, 0x7f, 0x37, + 0x88, 0x11, 0xf1, 0x14, 0xde, 0x52, 0x42, 0x64, 0x48, 0x4c, 0x26, 0xa2, 0x49, 0xb7, 0x45, 0x31, + 0x36, 0x4b, 0x42, 0x28, 0xe9, 0x69, 0x51, 0x80, 0xa7, 0x94, 0x64, 0xb7, 0xb9, 0x5c, 0xe0, 0x19, + 0x46, 0x5e, 0x14, 0x36, 0x43, 0xb1, 0xd7, 0xa2, 0xc9, 0x6d, 0x15, 0xca, 0x48, 0xc0, 0x44, 0x8d, + 0x48, 0xbd, 0x16, 0x17, 0x02, 0x99, 0x79, 0xf2, 0x48, 0x95, 0x97, 0xd0, 0x31, 0x3d, 0x7e, 0xcf, + 0x85, 0x3f, 0xa1, 0xc7, 0xaa, 0x8e, 0x81, 0xf8, 0xcd, 0x5a, 0x99, 0x0a, 0x32, 0xd4, 0xca, 0x3d, + 0xd9, 0x73, 0xe3, 0x05, 0x1d, 0x97, 0x92, 0xfd, 0x35, 0x91, 0x70, 0x95, 0x3c, 0x3d, 0x39, 0xf9, + 0x30, 0x37, 0xd9, 0x95, 0xfa, 0xa4, 0x1a, 0xdb, 0xf9, 0xf4, 0x1a, 0xdb, 0xed, 0x24, 0x63, 0x65, + 0x21, 0x53, 0xb8, 0x26, 0x98, 0x23, 0xf7, 0xe0, 0x4a, 0x5c, 0x5d, 0xee, 0xe2, 0x41, 0xcf, 0x1e, + 0x1c, 0xd2, 0x5c, 0x48, 0xd1, 0x48, 0x07, 0x40, 0x5b, 0x70, 0x4d, 0x51, 0xdd, 0x54, 0x99, 0x4b, + 0x6e, 0x17, 0xbb, 0xc7, 0x3f, 0x12, 0x86, 0xb8, 0xdb, 0x09, 0x13, 0x88, 0x68, 0x31, 0xbb, 0xcf, + 0x3f, 0x02, 0x02, 0x7d, 0x00, 0x57, 0xe3, 0xbd, 0xc1, 0xcd, 0x27, 0x91, 0x54, 0x19, 0x01, 0x72, + 0x66, 0xe3, 0xe0, 0x0f, 0x35, 0x98, 0x93, 0x1d, 0x99, 0x44, 0x57, 0xfa, 0x1a, 0xcc, 0xb2, 0x94, + 0xa7, 0xa8, 0xde, 0x99, 0x35, 0xc2, 0x06, 0x54, 0x86, 0x19, 0xd5, 0x22, 0x10, 0x8f, 0x52, 0x66, + 0xaa, 0xa0, 0x64, 0xa6, 0x56, 0xa1, 0x58, 0x77, 0x3e, 0x19, 0xd0, 0x9e, 0x29, 0xda, 0x13, 0x3c, + 0x57, 0xfe, 0xb4, 0x02, 0xba, 0xe0, 0xed, 0xe0, 0x06, 0x5e, 0x70, 0xdf, 0x4e, 0x93, 0xef, 0xdb, + 0x25, 0x85, 0x8b, 0x42, 0x73, 0x2e, 0xaf, 0x98, 0x73, 0x3b, 0x2a, 0xab, 0x31, 0x27, 0xf1, 0x33, + 0x49, 0x02, 0x25, 0xb8, 0x58, 0x37, 0x9a, 0xdd, 0x92, 0xbe, 0xd4, 0xf3, 0x3f, 0x2e, 0xaf, 0x7a, + 0xa0, 0x47, 0xaa, 0x2b, 0x44, 0xa2, 0xf5, 0xce, 0xc8, 0x57, 0x8d, 0x22, 0xc9, 0xea, 0x33, 0x36, + 0x22, 0x6a, 0xca, 0xee, 0x1a, 0xfb, 0x8c, 0xe2, 0xa7, 0x47, 0x0e, 0x1f, 0x40, 0xb3, 0x7d, 0x0c, + 0xb1, 0x65, 0xb5, 0x00, 0x63, 0xab, 0x05, 0x49, 0x71, 0x95, 0x4e, 0xa5, 0xb8, 0xe6, 0x26, 0x50, + 0x5c, 0x11, 0x35, 0x3b, 0x3f, 0xb1, 0x9a, 0x8d, 0xe9, 0x90, 0x85, 0x53, 0xe9, 0x10, 0x55, 0xbc, + 0x5f, 0x9c, 0x50, 0xbc, 0xc7, 0x62, 0x1c, 0xfa, 0x69, 0x62, 0x1c, 0x29, 0xc2, 0x7e, 0x71, 0x42, + 0x61, 0x8f, 0xce, 0x5d, 0xd8, 0x2f, 0x9d, 0x55, 0xd8, 0x2f, 0x9f, 0x59, 0xd8, 0xaf, 0x9c, 0x55, + 0xd8, 0x5f, 0xca, 0x14, 0xf6, 0xe8, 0xdd, 0x58, 0x2d, 0xa4, 0xa8, 0x00, 0x62, 0x39, 0xe2, 0x94, + 0xde, 0x04, 0x77, 0x24, 0xac, 0x2b, 0x2a, 0x27, 0xba, 0x23, 0x61, 0x99, 0xd1, 0x57, 0x61, 0x39, + 0xd2, 0x27, 0xf2, 0xc1, 0x31, 0x87, 0x38, 0xc6, 0xf7, 0x49, 0x88, 0x4c, 0x04, 0x24, 0x8e, 0x89, + 0x86, 0xb1, 0xf7, 0xab, 0xb5, 0x45, 0xfe, 0x38, 0x16, 0xcc, 0xc8, 0x9a, 0x8d, 0xa3, 0xb2, 0xf9, + 0x52, 0xc6, 0x4d, 0x98, 0xd1, 0x6c, 0x8b, 0xa4, 0xf3, 0xc4, 0x33, 0x9a, 0xa3, 0x66, 0xe4, 0x9d, + 0x68, 0x1b, 0x6e, 0x44, 0x7a, 0x78, 0xe1, 0x9c, 0x57, 0xf5, 0x3c, 0xfb, 0x70, 0x10, 0x7e, 0xe6, + 0x24, 0x03, 0x0c, 0xb5, 0xe0, 0xb5, 0xe8, 0x5b, 0x05, 0x05, 0x74, 0xc1, 0x58, 0x2c, 0x63, 0x9d, + 0x0d, 0x98, 0xea, 0x76, 0x86, 0x94, 0xb2, 0x36, 0xc2, 0xed, 0x0c, 0xe9, 0x65, 0x2b, 0xa5, 0x5e, + 0x4b, 0x50, 0xea, 0x8d, 0x78, 0x35, 0x43, 0x14, 0x26, 0x35, 0x8b, 0xc1, 0x62, 0xd9, 0xeb, 0x94, + 0x57, 0x47, 0x40, 0xa0, 0x47, 0xb0, 0x9e, 0x58, 0xe9, 0x20, 0x97, 0xbd, 0xbd, 0x46, 0xd7, 0x91, + 0x09, 0x37, 0x46, 0x95, 0x47, 0x65, 0xac, 0x2a, 0x8f, 0xaf, 0x6b, 0x70, 0x3d, 0x71, 0xc9, 0x34, + 0xd4, 0x41, 0x28, 0xee, 0x75, 0x4a, 0x71, 0xef, 0x8f, 0xa4, 0xb8, 0x91, 0x23, 0x30, 0xc2, 0x1b, + 0x3d, 0x0b, 0xfa, 0xed, 0xb4, 0x82, 0x3a, 0xc1, 0x6a, 0x37, 0xe9, 0x32, 0xee, 0x4f, 0xbe, 0x0c, + 0x85, 0xe1, 0x46, 0xce, 0x81, 0xbe, 0xa1, 0xa5, 0xa4, 0x3d, 0xa8, 0xca, 0x62, 0xeb, 0xf8, 0x14, + 0x5d, 0x47, 0x75, 0xf2, 0x75, 0x84, 0x63, 0xb0, 0xa5, 0x64, 0xcd, 0x84, 0x7e, 0x4f, 0x4b, 0xa1, + 0xfd, 0x5a, 0x5b, 0x7c, 0xf4, 0xe8, 0x0d, 0xba, 0x98, 0x0f, 0x4e, 0xb3, 0x29, 0x7c, 0x08, 0xb6, + 0x96, 0x8c, 0x79, 0xd0, 0x37, 0x35, 0x78, 0x2d, 0x7d, 0xb9, 0x62, 0x35, 0x6f, 0xd2, 0xd5, 0xd4, + 0x4e, 0xb9, 0x35, 0xca, 0x82, 0xb2, 0x67, 0x4b, 0xe5, 0x68, 0xa1, 0x7c, 0x6f, 0x8d, 0xe0, 0x68, + 0xa1, 0x7f, 0xff, 0x58, 0x83, 0xca, 0xc8, 0x57, 0x67, 0x45, 0x96, 0x6f, 0xd1, 0x17, 0xab, 0x9f, + 0x7e, 0x9b, 0xe9, 0x30, 0xec, 0xcd, 0xc6, 0x98, 0x0f, 0x7d, 0x5b, 0x83, 0x4f, 0x65, 0x6d, 0x00, + 0x5b, 0xd9, 0x06, 0x5d, 0xd9, 0xc3, 0x33, 0x6d, 0xb9, 0xb4, 0xb8, 0xf1, 0x66, 0x3d, 0x73, 0x00, + 0xfd, 0x63, 0x58, 0x49, 0x34, 0xed, 0x27, 0x8c, 0x53, 0x29, 0xf7, 0x0f, 0xa5, 0xe1, 0xef, 0xd1, + 0xaf, 0x5e, 0xa6, 0x04, 0xd5, 0x32, 0x17, 0xf7, 0x10, 0xae, 0xa4, 0x1a, 0x08, 0x59, 0x03, 0x15, + 0xe5, 0x81, 0x9a, 0xb1, 0x02, 0x0a, 0x59, 0x14, 0x9d, 0x71, 0x28, 0xf3, 0xb4, 0x43, 0xed, 0xa6, + 0x50, 0xbc, 0x22, 0xad, 0x27, 0x1a, 0x71, 0x27, 0x45, 0x36, 0x9c, 0xfa, 0x6d, 0x0d, 0xb8, 0x39, + 0x8e, 0x04, 0x9d, 0x68, 0xcc, 0x0f, 0xe1, 0xf5, 0x31, 0x04, 0xe1, 0x44, 0x84, 0x62, 0xc2, 0x1b, + 0xe3, 0x49, 0xb3, 0x89, 0x46, 0xdd, 0x83, 0x37, 0xc7, 0x14, 0x25, 0x13, 0x0d, 0xfb, 0x25, 0xd8, + 0x18, 0x5f, 0x0e, 0x4c, 0x14, 0xaa, 0x69, 0xb2, 0x5a, 0x24, 0xf1, 0x81, 0xd9, 0x33, 0x7c, 0x15, + 0xab, 0xf2, 0x97, 0x39, 0x58, 0x4e, 0xba, 0x9a, 0x3b, 0xe2, 0x86, 0xcc, 0x6e, 0xec, 0x7b, 0xc8, + 0x9b, 0x59, 0x17, 0x7d, 0xd5, 0xef, 0x22, 0xc7, 0x92, 0x38, 0xe7, 0xf2, 0x75, 0xe4, 0x55, 0x33, + 0xfb, 0x53, 0xc2, 0xa3, 0x8a, 0x95, 0xa4, 0x1d, 0x95, 0xf7, 0xfa, 0x3b, 0x1a, 0xc0, 0x96, 0xd5, + 0x3d, 0x3a, 0x19, 0xd2, 0x3c, 0x50, 0x5a, 0x92, 0xb0, 0x99, 0x94, 0x24, 0x7c, 0x53, 0xb9, 0x15, + 0x14, 0x0c, 0x32, 0x3a, 0x9e, 0x74, 0xe6, 0x40, 0xde, 0xef, 0x68, 0x22, 0xc7, 0xd5, 0xf4, 0xf1, + 0x71, 0xe2, 0x07, 0x7a, 0x2a, 0x30, 0xc7, 0xef, 0x1f, 0x3c, 0x95, 0x32, 0xa5, 0x4a, 0x1b, 0x81, + 0xa9, 0xe3, 0x67, 0xd6, 0x49, 0x9f, 0xc3, 0xf0, 0x2f, 0xe8, 0xca, 0x6d, 0xe4, 0x88, 0x9a, 0x03, + 0x1f, 0xbb, 0x03, 0xab, 0xcf, 0x93, 0x7b, 0xc1, 0x73, 0xe5, 0xaf, 0x34, 0x39, 0xd5, 0x86, 0xbe, + 0x08, 0x33, 0x35, 0x67, 0xe0, 0x63, 0xfa, 0x1d, 0x9b, 0x78, 0xc1, 0x7f, 0x00, 0xb8, 0xc9, 0xa1, + 0xd8, 0xc6, 0x08, 0x9c, 0x55, 0x83, 0xde, 0x43, 0x0d, 0x3a, 0x26, 0x2c, 0x1f, 0x0a, 0xb7, 0x43, + 0xde, 0xa8, 0xff, 0x1f, 0xe6, 0xf4, 0xd0, 0x3b, 0xe1, 0x2d, 0xed, 0x58, 0x95, 0x9c, 0x00, 0xda, + 0xa4, 0x10, 0x6c, 0x61, 0x0c, 0x7a, 0xf5, 0x3d, 0x80, 0xb0, 0x71, 0xa2, 0x5c, 0xf4, 0x9b, 0xca, + 0xc7, 0x21, 0x46, 0xdc, 0x5e, 0xfb, 0x18, 0x16, 0x63, 0xf7, 0xa4, 0xd0, 0x4d, 0x98, 0x67, 0xdf, + 0x9b, 0x12, 0x57, 0xaf, 0x18, 0x92, 0xda, 0x48, 0x8f, 0x99, 0xa3, 0x48, 0xdf, 0x28, 0x53, 0xda, + 0x36, 0x3e, 0x01, 0xd8, 0x1b, 0xf6, 0x2c, 0x9f, 0x45, 0x70, 0x2f, 0xc3, 0x92, 0xf2, 0xfd, 0x2a, + 0xd6, 0xa5, 0x5f, 0x40, 0x2b, 0xb0, 0x28, 0xbe, 0x4b, 0xd6, 0xea, 0xb4, 0x79, 0xb3, 0x86, 0x96, + 0xe0, 0xe2, 0x9e, 0x87, 0x5d, 0xfa, 0xfa, 0xbc, 0x31, 0x87, 0xe6, 0x61, 0xd6, 0xec, 0xec, 0xf0, + 0xc7, 0x3c, 0x41, 0x0d, 0xbe, 0x31, 0x16, 0xa0, 0x16, 0x36, 0x36, 0x61, 0x36, 0xf8, 0x36, 0x3d, + 0xba, 0x08, 0xa5, 0xb6, 0xe3, 0x1e, 0x5b, 0x7d, 0xfa, 0xa8, 0x5f, 0x40, 0x3a, 0xcc, 0xf1, 0x6b, + 0x35, 0xac, 0x45, 0xdb, 0xf8, 0x69, 0x01, 0x20, 0xfc, 0xae, 0x03, 0x5a, 0x00, 0x30, 0x3b, 0x3b, + 0xfb, 0x7b, 0xbb, 0xf5, 0xaa, 0xd9, 0xd0, 0x2f, 0x20, 0x80, 0xe9, 0xea, 0xee, 0x6e, 0xa3, 0x5d, + 0xd7, 0x35, 0x54, 0x84, 0x82, 0xd1, 0xa8, 0xd6, 0xf5, 0x1c, 0x9a, 0x83, 0xa2, 0x69, 0xec, 0xb5, + 0x6b, 0x04, 0x26, 0x4f, 0x06, 0x7d, 0xd8, 0x30, 0xf7, 0x83, 0x96, 0x02, 0x2a, 0xc1, 0x4c, 0x6d, + 0xa7, 0xdd, 0x6e, 0xd4, 0x4c, 0x7d, 0x8a, 0x0c, 0xc9, 0x1f, 0xf6, 0x8d, 0x1d, 0x7d, 0x1a, 0x2d, + 0xc2, 0x7c, 0x6b, 0xe7, 0xe1, 0xfe, 0x76, 0xa3, 0x6a, 0x98, 0x5b, 0x8d, 0xaa, 0xa9, 0xcf, 0x90, + 0x11, 0x6a, 0x6d, 0xa9, 0xa5, 0x48, 0x17, 0x2a, 0xb7, 0xcc, 0x22, 0x04, 0x0b, 0xb5, 0xed, 0x46, + 0xed, 0xf1, 0xfe, 0x76, 0xf5, 0x71, 0xa3, 0xb1, 0xdb, 0x30, 0x74, 0x20, 0xfb, 0x4a, 0x66, 0xae, + 0xb5, 0xf6, 0x3a, 0x66, 0xc3, 0xd8, 0xaf, 0x37, 0xcc, 0x6a, 0xb3, 0xd5, 0xd1, 0x4b, 0x04, 0x98, + 0x74, 0x74, 0xb6, 0xab, 0x46, 0x7d, 0xbf, 0xd9, 0x7e, 0xb0, 0xa3, 0xcf, 0xd1, 0x01, 0xda, 0xfb, + 0xd5, 0x56, 0x6b, 0x87, 0xac, 0x72, 0xbf, 0x59, 0xd7, 0xe7, 0xc9, 0x26, 0xca, 0x03, 0x74, 0x4c, + 0xb2, 0xfe, 0x05, 0xba, 0xff, 0x74, 0x07, 0xf6, 0x6b, 0xed, 0xfd, 0x56, 0x75, 0xab, 0xd1, 0xd2, + 0x2f, 0xa2, 0x32, 0x2c, 0x87, 0x8d, 0x1f, 0xed, 0x18, 0x8f, 0x39, 0xb8, 0x4e, 0x46, 0xde, 0xad, + 0x9a, 0xb5, 0x6d, 0xd2, 0xd1, 0x31, 0x77, 0x8c, 0x86, 0xbe, 0x48, 0x86, 0xa8, 0x37, 0x5a, 0x0d, + 0x06, 0xcd, 0x1a, 0x11, 0x69, 0xdc, 0x35, 0x76, 0xbe, 0xf4, 0x65, 0xe9, 0xc5, 0x96, 0xd0, 0x6b, + 0x70, 0x9d, 0x8f, 0xdb, 0xde, 0x69, 0xef, 0x3f, 0xdd, 0x31, 0x9b, 0xed, 0x87, 0xfb, 0x46, 0x63, + 0xb7, 0xd5, 0xac, 0x55, 0xf7, 0xdb, 0x7b, 0x4f, 0xf4, 0x65, 0xb4, 0x06, 0xab, 0x71, 0x10, 0xf2, + 0x1e, 0xad, 0xa6, 0xf9, 0x65, 0x7d, 0x05, 0x2d, 0x83, 0xde, 0x69, 0x98, 0xfb, 0x46, 0xe3, 0xc3, + 0xbd, 0xa6, 0xd1, 0xa8, 0xef, 0xb7, 0x3a, 0x6d, 0xfd, 0x12, 0x69, 0x7d, 0x18, 0x6d, 0xbd, 0x2c, + 0xb6, 0xa6, 0x55, 0x35, 0x1b, 0x1d, 0x93, 0xb6, 0x95, 0xc9, 0x91, 0xd0, 0xb6, 0x46, 0xb5, 0xde, + 0x30, 0xc8, 0xce, 0x5c, 0xa1, 0x47, 0xc2, 0xb6, 0xbb, 0x51, 0x6d, 0x99, 0xdb, 0xfa, 0x2a, 0x39, + 0x74, 0x72, 0xfc, 0x14, 0xe5, 0x2a, 0xba, 0x02, 0x2b, 0x7c, 0x49, 0xad, 0x46, 0xb5, 0xd3, 0xd8, + 0xde, 0x69, 0x71, 0xd4, 0x6b, 0xa4, 0x8b, 0x6e, 0x7e, 0x6d, 0xbb, 0x51, 0xdf, 0x6b, 0x35, 0xf6, + 0x6b, 0x3b, 0x4f, 0x9e, 0x54, 0xdb, 0xf5, 0x8e, 0x7e, 0x7d, 0xa3, 0x0d, 0x10, 0x7e, 0x0d, 0x8d, + 0x50, 0x06, 0x21, 0x73, 0xd6, 0xa2, 0x5f, 0x20, 0x33, 0x08, 0x39, 0xa7, 0x6b, 0x84, 0x78, 0x29, + 0xd3, 0x04, 0x0c, 0xb0, 0xc8, 0x3f, 0xff, 0x67, 0xe0, 0xaf, 0xe2, 0xae, 0x8f, 0x7b, 0x7a, 0x7e, + 0x63, 0x03, 0x66, 0x83, 0x8f, 0x6d, 0x11, 0xf4, 0x0e, 0xf6, 0xe9, 0x93, 0x7e, 0x81, 0xa0, 0xb3, + 0xe0, 0x2a, 0x6b, 0xd0, 0x36, 0xfe, 0xb1, 0x00, 0x48, 0xb8, 0x14, 0x12, 0x6f, 0x12, 0x8a, 0xb7, + 0xbb, 0x47, 0x32, 0x4b, 0x4a, 0x5f, 0x35, 0x0a, 0x58, 0x92, 0x70, 0x6a, 0xac, 0x39, 0x87, 0x2e, + 0xd1, 0x5a, 0x93, 0x68, 0x7b, 0x9e, 0xcc, 0xfe, 0x10, 0xfb, 0x01, 0xa7, 0x17, 0xc8, 0xa6, 0x44, + 0xe4, 0x0d, 0xef, 0x9a, 0x22, 0x27, 0xd2, 0xc1, 0x8c, 0x21, 0x79, 0xdb, 0x34, 0x21, 0x36, 0xb5, + 0x98, 0x88, 0xf7, 0xcc, 0xa0, 0x1b, 0x70, 0xb5, 0x83, 0xfd, 0x78, 0x6e, 0x82, 0x03, 0x14, 0xd1, + 0x2a, 0x5c, 0xe2, 0x00, 0x41, 0x70, 0x9b, 0xf7, 0xcd, 0x92, 0x2d, 0x64, 0xbf, 0xf9, 0xae, 0xe9, + 0x40, 0x5e, 0x4c, 0x34, 0x05, 0x17, 0xba, 0xf4, 0x12, 0x39, 0xff, 0x5d, 0x22, 0xef, 0x78, 0xfd, + 0xa0, 0x3e, 0x47, 0x70, 0x0d, 0x7c, 0xec, 0xbc, 0x10, 0x37, 0x8e, 0xf5, 0x79, 0xb2, 0x4a, 0xb5, + 0x50, 0x94, 0x4f, 0xb4, 0x80, 0xae, 0xc3, 0x15, 0xf6, 0x3b, 0x21, 0x68, 0xad, 0x5f, 0x44, 0x57, + 0xe1, 0x72, 0xa4, 0x5b, 0x68, 0x03, 0xc6, 0x4e, 0xc2, 0xeb, 0xe1, 0xe3, 0x2d, 0xa2, 0x6b, 0x50, + 0x8e, 0x97, 0x03, 0xf1, 0x5e, 0x84, 0x6e, 0xc2, 0xba, 0x08, 0xe2, 0xc6, 0xc3, 0xbb, 0x1c, 0x6a, + 0x89, 0xec, 0x1c, 0x0b, 0x80, 0x45, 0x3c, 0x10, 0x0e, 0xb0, 0x8c, 0x3e, 0x05, 0xaf, 0x31, 0x80, + 0x44, 0x03, 0x93, 0x83, 0xad, 0x6c, 0x7c, 0x4b, 0x83, 0x79, 0x25, 0xdb, 0x44, 0xf8, 0x5a, 0x34, + 0xf0, 0x8a, 0x18, 0xfd, 0x02, 0x39, 0x71, 0xd1, 0xa8, 0x7c, 0x37, 0x42, 0xd7, 0xc8, 0x44, 0xb1, + 0x2e, 0xe1, 0x3f, 0x1a, 0xb8, 0x8b, 0xed, 0x17, 0xb8, 0xa7, 0xe7, 0xc8, 0x2e, 0xc5, 0xc0, 0x1e, + 0x58, 0x76, 0x9f, 0x90, 0xbe, 0x3c, 0xa7, 0x71, 0x32, 0x18, 0x90, 0x81, 0x0b, 0x1b, 0x07, 0x49, + 0xf9, 0x2e, 0x72, 0x4c, 0x4a, 0x6b, 0xb8, 0xc6, 0x68, 0x8f, 0x18, 0x49, 0x8b, 0xf5, 0x74, 0x7c, + 0x67, 0x38, 0x24, 0xab, 0xda, 0xf8, 0x81, 0x06, 0x7a, 0xf4, 0x4b, 0x21, 0x84, 0x8b, 0xaa, 0x3d, + 0xf1, 0xf1, 0x3e, 0xfd, 0x42, 0x48, 0x2c, 0xa2, 0x49, 0x23, 0x14, 0xd5, 0xf1, 0x2d, 0xd7, 0x17, + 0x2d, 0x39, 0xc2, 0x24, 0x64, 0x58, 0xd1, 0x90, 0x27, 0xa3, 0x3c, 0xb6, 0xfb, 0xfd, 0xaf, 0x38, + 0xc7, 0x07, 0x36, 0x61, 0x9a, 0xcb, 0xb0, 0x54, 0xed, 0xf5, 0xa2, 0x24, 0xa4, 0x4f, 0x11, 0x1a, + 0x67, 0xc3, 0xc7, 0xfa, 0xa6, 0x29, 0xa7, 0x91, 0x79, 0x62, 0x5d, 0x33, 0xe4, 0xa5, 0xc8, 0x84, + 0xb1, 0x9e, 0xe2, 0xc6, 0x13, 0xe5, 0x0b, 0x0a, 0x64, 0x21, 0x21, 0x21, 0xe9, 0x17, 0xa8, 0xee, + 0x6d, 0x8b, 0x47, 0x8d, 0x3c, 0xd6, 0x82, 0xc7, 0x1c, 0xe5, 0x15, 0x9a, 0x0a, 0xe2, 0x2d, 0xf9, + 0xad, 0xe6, 0xf7, 0x7f, 0xbc, 0x76, 0xe1, 0x7b, 0x3f, 0x59, 0xd3, 0xbe, 0xff, 0x93, 0x35, 0xed, + 0x47, 0x3f, 0x59, 0xbb, 0xf0, 0xd7, 0xff, 0xb6, 0xa6, 0x7d, 0xe5, 0xae, 0xf4, 0xbf, 0xe8, 0x8e, + 0x2d, 0xdf, 0xb5, 0x5f, 0x3a, 0xd4, 0xb0, 0x10, 0x0f, 0x03, 0x7c, 0x7b, 0x78, 0x74, 0x78, 0x7b, + 0x78, 0x70, 0x3b, 0x34, 0x93, 0x0e, 0xa6, 0xe9, 0x97, 0x16, 0xef, 0xfe, 0x77, 0x00, 0x00, 0x00, + 0xff, 0xff, 0xd7, 0xd9, 0xa2, 0x6b, 0xe7, 0x6e, 0x00, 0x00, } func (m *CNStore) Marshal() (dAtA []byte, err error) { @@ -6898,6 +7209,73 @@ func (m *CNStore) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + { + size, err := m.DDLVisibilityFrontier.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xd2 + if m.DDLVisibilityActivationFenced { + i-- + if m.DDLVisibilityActivationFenced { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xc8 + } + if m.DDLVisibilityActivationPrepared { + i-- + if m.DDLVisibilityActivationPrepared { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xc0 + } + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xb8 + } + if m.ViewMetadataIngressReady { + i-- + if m.ViewMetadataIngressReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xb0 + } + if m.DDLVisibilityBarrierReady { + i-- + if m.DDLVisibilityBarrierReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xa8 + } if m.ViewMetadataAdmissionReady { i-- if m.ViewMetadataAdmissionReady { @@ -7200,6 +7578,16 @@ func (m *LogStore) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.DDLVisibilityDeployedProtocolSupported { + i-- + if m.DDLVisibilityDeployedProtocolSupported { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x40 + } { size, err := m.Locality.MarshalToSizedBuffer(dAtA[:i]) if err != nil { @@ -7461,6 +7849,52 @@ func (m *Resource) MarshalToSizedBuffer(dAtA []byte) (int, error) { return len(dAtA) - i, nil } +func (m *DDLVisibilityActivationTarget) Marshal() (dAtA []byte, err error) { + size := m.ProtoSize() + dAtA = make([]byte, size) + n, err := m.MarshalToSizedBuffer(dAtA[:size]) + if err != nil { + return nil, err + } + return dAtA[:n], nil +} + +func (m *DDLVisibilityActivationTarget) MarshalTo(dAtA []byte) (int, error) { + size := m.ProtoSize() + return m.MarshalToSizedBuffer(dAtA[:size]) +} + +func (m *DDLVisibilityActivationTarget) MarshalToSizedBuffer(dAtA []byte) (int, error) { + i := len(dAtA) + _ = i + var l int + _ = l + if m.XXX_unrecognized != nil { + i -= len(m.XXX_unrecognized) + copy(dAtA[i:], m.XXX_unrecognized) + } + if len(m.QueryAddress) > 0 { + i -= len(m.QueryAddress) + copy(dAtA[i:], m.QueryAddress) + i = encodeVarintLogservice(dAtA, i, uint64(len(m.QueryAddress))) + i-- + dAtA[i] = 0x1a + } + if m.Generation != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.Generation)) + i-- + dAtA[i] = 0x10 + } + if len(m.ServiceID) > 0 { + i -= len(m.ServiceID) + copy(dAtA[i:], m.ServiceID) + i = encodeVarintLogservice(dAtA, i, uint64(len(m.ServiceID))) + i-- + dAtA[i] = 0xa + } + return len(dAtA) - i, nil +} + func (m *CNStoreHeartbeat) Marshal() (dAtA []byte, err error) { size := m.ProtoSize() dAtA = make([]byte, size) @@ -7485,6 +7919,77 @@ func (m *CNStoreHeartbeat) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + { + size, err := m.DDLVisibilityFrontier.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xf2 + if m.DDLVisibilityActivationFenced { + i-- + if m.DDLVisibilityActivationFenced { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xe8 + } + if m.DDLVisibilityActivationPrepared { + i-- + if m.DDLVisibilityActivationPrepared { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xe0 + } + if len(m.DDLVisibilityEpochCommitTargets) > 0 { + for iNdEx := len(m.DDLVisibilityEpochCommitTargets) - 1; iNdEx >= 0; iNdEx-- { + { + size, err := m.DDLVisibilityEpochCommitTargets[iNdEx].MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xda + } + } + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xd0 + } + if m.DDLVisibilityBarrierReady { + i-- + if m.DDLVisibilityBarrierReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xc8 + } if m.ViewMetadataIngressReady { i-- if m.ViewMetadataIngressReady { @@ -7751,6 +8256,16 @@ func (m *LogStoreHeartbeat) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.DDLVisibilityDeployedProtocolSupported { + i-- + if m.DDLVisibilityDeployedProtocolSupported { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x60 + } if len(m.StoreIncarnation) > 0 { i -= len(m.StoreIncarnation) copy(dAtA[i:], m.StoreIncarnation) @@ -8224,12 +8739,12 @@ func (m *LogRequest) MarshalToSizedBuffer(dAtA []byte) (int, error) { i-- dAtA[i] = 0x40 } - n13, err13 := github_com_gogo_protobuf_types.StdTimeMarshalTo(m.TS, dAtA[i-github_com_gogo_protobuf_types.SizeOfStdTime(m.TS):]) - if err13 != nil { - return 0, err13 + n15, err15 := github_com_gogo_protobuf_types.StdTimeMarshalTo(m.TS, dAtA[i-github_com_gogo_protobuf_types.SizeOfStdTime(m.TS):]) + if err15 != nil { + return 0, err15 } - i -= n13 - i = encodeVarintLogservice(dAtA, i, uint64(n13)) + i -= n15 + i = encodeVarintLogservice(dAtA, i, uint64(n15)) i-- dAtA[i] = 0x3a if m.TNID != 0 { @@ -9684,6 +10199,23 @@ func (m *CommandBatch) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.DDLVisibilityFrontier != nil { + { + size, err := m.DDLVisibilityFrontier.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x3a + } + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) + i-- + dAtA[i] = 0x30 + } if m.ViewMetadataAdmission != nil { { size, err := m.ViewMetadataAdmission.MarshalToSizedBuffer(dAtA[:i]) @@ -9910,9 +10442,21 @@ func (m *CNStoreInfo) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } - if m.ViewMetadataIngressReady { + { + size, err := m.DDLVisibilityFrontier.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xfa + if m.DDLVisibilityActivationFenced { i-- - if m.ViewMetadataIngressReady { + if m.DDLVisibilityActivationFenced { dAtA[i] = 1 } else { dAtA[i] = 0 @@ -9920,11 +10464,11 @@ func (m *CNStoreInfo) MarshalToSizedBuffer(dAtA []byte) (int, error) { i-- dAtA[i] = 0x1 i-- - dAtA[i] = 0xd0 + dAtA[i] = 0xf0 } - if m.ViewMetadataAdmissionReady { + if m.DDLVisibilityActivationPrepared { i-- - if m.ViewMetadataAdmissionReady { + if m.DDLVisibilityActivationPrepared { dAtA[i] = 1 } else { dAtA[i] = 0 @@ -9932,18 +10476,18 @@ func (m *CNStoreInfo) MarshalToSizedBuffer(dAtA []byte) (int, error) { i-- dAtA[i] = 0x1 i-- - dAtA[i] = 0xc8 + dAtA[i] = 0xe8 } - if m.ViewMetadataRevalidatedEpoch != 0 { - i = encodeVarintLogservice(dAtA, i, uint64(m.ViewMetadataRevalidatedEpoch)) + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) i-- dAtA[i] = 0x1 i-- - dAtA[i] = 0xc0 + dAtA[i] = 0xe0 } - if m.ViewMetadataRefreshSupported { + if m.DDLVisibilityBarrierReady { i-- - if m.ViewMetadataRefreshSupported { + if m.DDLVisibilityBarrierReady { dAtA[i] = 1 } else { dAtA[i] = 0 @@ -9951,19 +10495,62 @@ func (m *CNStoreInfo) MarshalToSizedBuffer(dAtA []byte) (int, error) { i-- dAtA[i] = 0x1 i-- - dAtA[i] = 0xb8 + dAtA[i] = 0xd8 } - if m.ViewMetadataCatalogFencedEpoch != 0 { - i = encodeVarintLogservice(dAtA, i, uint64(m.ViewMetadataCatalogFencedEpoch)) + if m.ViewMetadataIngressReady { + i-- + if m.ViewMetadataIngressReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } i-- dAtA[i] = 0x1 i-- - dAtA[i] = 0xb0 + dAtA[i] = 0xd0 } - if m.ViewMetadataObservedEpoch != 0 { - i = encodeVarintLogservice(dAtA, i, uint64(m.ViewMetadataObservedEpoch)) + if m.ViewMetadataAdmissionReady { i-- - dAtA[i] = 0x1 + if m.ViewMetadataAdmissionReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xc8 + } + if m.ViewMetadataRevalidatedEpoch != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.ViewMetadataRevalidatedEpoch)) + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xc0 + } + if m.ViewMetadataRefreshSupported { + i-- + if m.ViewMetadataRefreshSupported { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xb8 + } + if m.ViewMetadataCatalogFencedEpoch != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.ViewMetadataCatalogFencedEpoch)) + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0xb0 + } + if m.ViewMetadataObservedEpoch != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.ViewMetadataObservedEpoch)) + i-- + dAtA[i] = 0x1 i-- dAtA[i] = 0xa8 } @@ -10164,6 +10751,21 @@ func (m *CNState) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + { + size, err := m.DDLVisibilityFrontier.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x1a + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) + i-- + dAtA[i] = 0x10 + } if len(m.Stores) > 0 { for k := range m.Stores { v := m.Stores[k] @@ -10606,6 +11208,21 @@ func (m *ClusterDetails) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + { + size, err := m.DDLVisibilityFrontier.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintLogservice(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x42 + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintLogservice(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) + i-- + dAtA[i] = 0x38 + } if m.ViewMetadataAdmission != nil { { size, err := m.ViewMetadataAdmission.MarshalToSizedBuffer(dAtA[:i]) @@ -10922,6 +11539,16 @@ func (m *LogStoreInfo) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.DDLVisibilityDeployedProtocolSupported { + i-- + if m.DDLVisibilityDeployedProtocolSupported { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x60 + } if len(m.StoreIncarnation) > 0 { i -= len(m.StoreIncarnation) copy(dAtA[i:], m.StoreIncarnation) @@ -12397,6 +13024,23 @@ func (m *CNStore) ProtoSize() (n int) { if m.ViewMetadataAdmissionReady { n += 3 } + if m.DDLVisibilityBarrierReady { + n += 3 + } + if m.ViewMetadataIngressReady { + n += 3 + } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 2 + sovLogservice(uint64(m.DDLVisibilityDeployedProtocol)) + } + if m.DDLVisibilityActivationPrepared { + n += 3 + } + if m.DDLVisibilityActivationFenced { + n += 3 + } + l = m.DDLVisibilityFrontier.ProtoSize() + n += 2 + l + sovLogservice(uint64(l)) if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -12490,6 +13134,9 @@ func (m *LogStore) ProtoSize() (n int) { } l = m.Locality.ProtoSize() n += 1 + l + sovLogservice(uint64(l)) + if m.DDLVisibilityDeployedProtocolSupported { + n += 2 + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -12588,6 +13235,29 @@ func (m *Resource) ProtoSize() (n int) { return n } +func (m *DDLVisibilityActivationTarget) ProtoSize() (n int) { + if m == nil { + return 0 + } + var l int + _ = l + l = len(m.ServiceID) + if l > 0 { + n += 1 + l + sovLogservice(uint64(l)) + } + if m.Generation != 0 { + n += 1 + sovLogservice(uint64(m.Generation)) + } + l = len(m.QueryAddress) + if l > 0 { + n += 1 + l + sovLogservice(uint64(l)) + } + if m.XXX_unrecognized != nil { + n += len(m.XXX_unrecognized) + } + return n +} + func (m *CNStoreHeartbeat) ProtoSize() (n int) { if m == nil { return 0 @@ -12672,6 +13342,26 @@ func (m *CNStoreHeartbeat) ProtoSize() (n int) { if m.ViewMetadataIngressReady { n += 3 } + if m.DDLVisibilityBarrierReady { + n += 3 + } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 2 + sovLogservice(uint64(m.DDLVisibilityDeployedProtocol)) + } + if len(m.DDLVisibilityEpochCommitTargets) > 0 { + for _, e := range m.DDLVisibilityEpochCommitTargets { + l = e.ProtoSize() + n += 2 + l + sovLogservice(uint64(l)) + } + } + if m.DDLVisibilityActivationPrepared { + n += 3 + } + if m.DDLVisibilityActivationFenced { + n += 3 + } + l = m.DDLVisibilityFrontier.ProtoSize() + n += 2 + l + sovLogservice(uint64(l)) if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -12748,6 +13438,9 @@ func (m *LogStoreHeartbeat) ProtoSize() (n int) { if l > 0 { n += 1 + l + sovLogservice(uint64(l)) } + if m.DDLVisibilityDeployedProtocolSupported { + n += 2 + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -13580,6 +14273,13 @@ func (m *CommandBatch) ProtoSize() (n int) { l = m.ViewMetadataAdmission.ProtoSize() n += 1 + l + sovLogservice(uint64(l)) } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 1 + sovLogservice(uint64(m.DDLVisibilityDeployedProtocol)) + } + if m.DDLVisibilityFrontier != nil { + l = m.DDLVisibilityFrontier.ProtoSize() + n += 1 + l + sovLogservice(uint64(l)) + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -13740,6 +14440,20 @@ func (m *CNStoreInfo) ProtoSize() (n int) { if m.ViewMetadataIngressReady { n += 3 } + if m.DDLVisibilityBarrierReady { + n += 3 + } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 2 + sovLogservice(uint64(m.DDLVisibilityDeployedProtocol)) + } + if m.DDLVisibilityActivationPrepared { + n += 3 + } + if m.DDLVisibilityActivationFenced { + n += 3 + } + l = m.DDLVisibilityFrontier.ProtoSize() + n += 2 + l + sovLogservice(uint64(l)) if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -13761,6 +14475,11 @@ func (m *CNState) ProtoSize() (n int) { n += mapEntrySize + 1 + sovLogservice(uint64(mapEntrySize)) } } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 1 + sovLogservice(uint64(m.DDLVisibilityDeployedProtocol)) + } + l = m.DDLVisibilityFrontier.ProtoSize() + n += 1 + l + sovLogservice(uint64(l)) if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -13978,6 +14697,11 @@ func (m *ClusterDetails) ProtoSize() (n int) { l = m.ViewMetadataAdmission.ProtoSize() n += 1 + l + sovLogservice(uint64(l)) } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 1 + sovLogservice(uint64(m.DDLVisibilityDeployedProtocol)) + } + l = m.DDLVisibilityFrontier.ProtoSize() + n += 1 + l + sovLogservice(uint64(l)) if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -14123,6 +14847,9 @@ func (m *LogStoreInfo) ProtoSize() (n int) { if l > 0 { n += 1 + l + sovLogservice(uint64(l)) } + if m.DDLVisibilityDeployedProtocolSupported { + n += 2 + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -15264,62 +15991,11 @@ func (m *CNStore) Unmarshal(dAtA []byte) error { } } m.ViewMetadataAdmissionReady = bool(v != 0) - default: - iNdEx = preIndex - skippy, err := skipLogservice(dAtA[iNdEx:]) - if err != nil { - return err - } - if (skippy < 0) || (iNdEx+skippy) < 0 { - return ErrInvalidLengthLogservice - } - if (iNdEx + skippy) > l { - return io.ErrUnexpectedEOF - } - m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...) - iNdEx += skippy - } - } - - if iNdEx > l { - return io.ErrUnexpectedEOF - } - return nil -} -func (m *TNStore) Unmarshal(dAtA []byte) error { - l := len(dAtA) - iNdEx := 0 - for iNdEx < l { - preIndex := iNdEx - var wire uint64 - for shift := uint(0); ; shift += 7 { - if shift >= 64 { - return ErrIntOverflowLogservice - } - if iNdEx >= l { - return io.ErrUnexpectedEOF - } - b := dAtA[iNdEx] - iNdEx++ - wire |= uint64(b&0x7F) << shift - if b < 0x80 { - break - } - } - fieldNum := int32(wire >> 3) - wireType := int(wire & 0x7) - if wireType == 4 { - return fmt.Errorf("proto: TNStore: wiretype end group for non-group") - } - if fieldNum <= 0 { - return fmt.Errorf("proto: TNStore: illegal tag %d (wire type %d)", fieldNum, wire) - } - switch fieldNum { - case 1: - if wireType != 2 { - return fmt.Errorf("proto: wrong wireType = %d for field UUID", wireType) + case 21: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityBarrierReady", wireType) } - var stringLen uint64 + var v int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -15329,29 +16005,17 @@ func (m *TNStore) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - stringLen |= uint64(b&0x7F) << shift + v |= int(b&0x7F) << shift if b < 0x80 { break } } - intStringLen := int(stringLen) - if intStringLen < 0 { - return ErrInvalidLengthLogservice - } - postIndex := iNdEx + intStringLen - if postIndex < 0 { - return ErrInvalidLengthLogservice - } - if postIndex > l { - return io.ErrUnexpectedEOF - } - m.UUID = string(dAtA[iNdEx:postIndex]) - iNdEx = postIndex - case 2: - if wireType != 2 { - return fmt.Errorf("proto: wrong wireType = %d for field ServiceAddress", wireType) + m.DDLVisibilityBarrierReady = bool(v != 0) + case 22: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field ViewMetadataIngressReady", wireType) } - var stringLen uint64 + var v int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -15361,29 +16025,17 @@ func (m *TNStore) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - stringLen |= uint64(b&0x7F) << shift + v |= int(b&0x7F) << shift if b < 0x80 { break } } - intStringLen := int(stringLen) - if intStringLen < 0 { - return ErrInvalidLengthLogservice - } - postIndex := iNdEx + intStringLen - if postIndex < 0 { - return ErrInvalidLengthLogservice - } - if postIndex > l { - return io.ErrUnexpectedEOF - } - m.ServiceAddress = string(dAtA[iNdEx:postIndex]) - iNdEx = postIndex - case 3: + m.ViewMetadataIngressReady = bool(v != 0) + case 23: if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field Tick", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) } - m.Tick = 0 + m.DDLVisibilityDeployedProtocol = 0 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -15393,16 +16045,16 @@ func (m *TNStore) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - m.Tick |= uint64(b&0x7F) << shift + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift if b < 0x80 { break } } - case 4: + case 24: if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field State", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationPrepared", wireType) } - m.State = 0 + var v int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -15412,16 +16064,17 @@ func (m *TNStore) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - m.State |= NodeState(b&0x7F) << shift + v |= int(b&0x7F) << shift if b < 0x80 { break } } - case 5: - if wireType != 2 { - return fmt.Errorf("proto: wrong wireType = %d for field Shards", wireType) + m.DDLVisibilityActivationPrepared = bool(v != 0) + case 25: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationFenced", wireType) } - var msglen int + var v int for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -15431,12 +16084,218 @@ func (m *TNStore) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - msglen |= int(b&0x7F) << shift + v |= int(b&0x7F) << shift if b < 0x80 { break } } - if msglen < 0 { + m.DDLVisibilityActivationFenced = bool(v != 0) + case 26: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityFrontier", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + if err := m.DDLVisibilityFrontier.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex + default: + iNdEx = preIndex + skippy, err := skipLogservice(dAtA[iNdEx:]) + if err != nil { + return err + } + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthLogservice + } + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF + } + m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...) + iNdEx += skippy + } + } + + if iNdEx > l { + return io.ErrUnexpectedEOF + } + return nil +} +func (m *TNStore) Unmarshal(dAtA []byte) error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: TNStore: wiretype end group for non-group") + } + if fieldNum <= 0 { + return fmt.Errorf("proto: TNStore: illegal tag %d (wire type %d)", fieldNum, wire) + } + switch fieldNum { + case 1: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field UUID", wireType) + } + var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.UUID = string(dAtA[iNdEx:postIndex]) + iNdEx = postIndex + case 2: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field ServiceAddress", wireType) + } + var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.ServiceAddress = string(dAtA[iNdEx:postIndex]) + iNdEx = postIndex + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field Tick", wireType) + } + m.Tick = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.Tick |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 4: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field State", wireType) + } + m.State = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.State |= NodeState(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 5: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Shards", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { return ErrInvalidLengthLogservice } postIndex := iNdEx + msglen @@ -15891,6 +16750,26 @@ func (m *LogStore) Unmarshal(dAtA []byte) error { return err } iNdEx = postIndex + case 8: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocolSupported", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityDeployedProtocolSupported = bool(v != 0) default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -16437,15 +17316,135 @@ func (m *LogReplicaInfo) Unmarshal(dAtA []byte) error { if postIndex > l { return io.ErrUnexpectedEOF } - if err := m.LogShardInfo.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { - return err - } - iNdEx = postIndex - case 2: + if err := m.LogShardInfo.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex + case 2: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field ReplicaID", wireType) + } + m.ReplicaID = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.ReplicaID |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field IsNonVoting", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.IsNonVoting = bool(v != 0) + default: + iNdEx = preIndex + skippy, err := skipLogservice(dAtA[iNdEx:]) + if err != nil { + return err + } + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthLogservice + } + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF + } + m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...) + iNdEx += skippy + } + } + + if iNdEx > l { + return io.ErrUnexpectedEOF + } + return nil +} +func (m *Resource) Unmarshal(dAtA []byte) error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: Resource: wiretype end group for non-group") + } + if fieldNum <= 0 { + return fmt.Errorf("proto: Resource: illegal tag %d (wire type %d)", fieldNum, wire) + } + switch fieldNum { + case 1: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field CPUTotal", wireType) + } + m.CPUTotal = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.CPUTotal |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 2: + if wireType != 1 { + return fmt.Errorf("proto: wrong wireType = %d for field CPUAvailable", wireType) + } + var v uint64 + if (iNdEx + 8) > l { + return io.ErrUnexpectedEOF + } + v = uint64(encoding_binary.LittleEndian.Uint64(dAtA[iNdEx:])) + iNdEx += 8 + m.CPUAvailable = float64(math.Float64frombits(v)) + case 3: if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field ReplicaID", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field MemTotal", wireType) } - m.ReplicaID = 0 + m.MemTotal = 0 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -16455,16 +17454,16 @@ func (m *LogReplicaInfo) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - m.ReplicaID |= uint64(b&0x7F) << shift + m.MemTotal |= uint64(b&0x7F) << shift if b < 0x80 { break } } - case 3: + case 4: if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field IsNonVoting", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field MemAvailable", wireType) } - var v int + m.MemAvailable = 0 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -16474,12 +17473,11 @@ func (m *LogReplicaInfo) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - v |= int(b&0x7F) << shift + m.MemAvailable |= uint64(b&0x7F) << shift if b < 0x80 { break } } - m.IsNonVoting = bool(v != 0) default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -16502,7 +17500,7 @@ func (m *LogReplicaInfo) Unmarshal(dAtA []byte) error { } return nil } -func (m *Resource) Unmarshal(dAtA []byte) error { +func (m *DDLVisibilityActivationTarget) Unmarshal(dAtA []byte) error { l := len(dAtA) iNdEx := 0 for iNdEx < l { @@ -16525,17 +17523,17 @@ func (m *Resource) Unmarshal(dAtA []byte) error { fieldNum := int32(wire >> 3) wireType := int(wire & 0x7) if wireType == 4 { - return fmt.Errorf("proto: Resource: wiretype end group for non-group") + return fmt.Errorf("proto: DDLVisibilityActivationTarget: wiretype end group for non-group") } if fieldNum <= 0 { - return fmt.Errorf("proto: Resource: illegal tag %d (wire type %d)", fieldNum, wire) + return fmt.Errorf("proto: DDLVisibilityActivationTarget: illegal tag %d (wire type %d)", fieldNum, wire) } switch fieldNum { case 1: - if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field CPUTotal", wireType) + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field ServiceID", wireType) } - m.CPUTotal = 0 + var stringLen uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -16545,27 +17543,29 @@ func (m *Resource) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - m.CPUTotal |= uint64(b&0x7F) << shift + stringLen |= uint64(b&0x7F) << shift if b < 0x80 { break } } - case 2: - if wireType != 1 { - return fmt.Errorf("proto: wrong wireType = %d for field CPUAvailable", wireType) + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthLogservice } - var v uint64 - if (iNdEx + 8) > l { + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { return io.ErrUnexpectedEOF } - v = uint64(encoding_binary.LittleEndian.Uint64(dAtA[iNdEx:])) - iNdEx += 8 - m.CPUAvailable = float64(math.Float64frombits(v)) - case 3: + m.ServiceID = string(dAtA[iNdEx:postIndex]) + iNdEx = postIndex + case 2: if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field MemTotal", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field Generation", wireType) } - m.MemTotal = 0 + m.Generation = 0 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -16575,16 +17575,16 @@ func (m *Resource) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - m.MemTotal |= uint64(b&0x7F) << shift + m.Generation |= uint64(b&0x7F) << shift if b < 0x80 { break } } - case 4: - if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field MemAvailable", wireType) + case 3: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field QueryAddress", wireType) } - m.MemAvailable = 0 + var stringLen uint64 for shift := uint(0); ; shift += 7 { if shift >= 64 { return ErrIntOverflowLogservice @@ -16594,11 +17594,24 @@ func (m *Resource) Unmarshal(dAtA []byte) error { } b := dAtA[iNdEx] iNdEx++ - m.MemAvailable |= uint64(b&0x7F) << shift + stringLen |= uint64(b&0x7F) << shift if b < 0x80 { break } } + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.QueryAddress = string(dAtA[iNdEx:postIndex]) + iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -17241,6 +18254,152 @@ func (m *CNStoreHeartbeat) Unmarshal(dAtA []byte) error { } } m.ViewMetadataIngressReady = bool(v != 0) + case 25: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityBarrierReady", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityBarrierReady = bool(v != 0) + case 26: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) + } + m.DDLVisibilityDeployedProtocol = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 27: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityEpochCommitTargets", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.DDLVisibilityEpochCommitTargets = append(m.DDLVisibilityEpochCommitTargets, DDLVisibilityActivationTarget{}) + if err := m.DDLVisibilityEpochCommitTargets[len(m.DDLVisibilityEpochCommitTargets)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex + case 28: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationPrepared", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityActivationPrepared = bool(v != 0) + case 29: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationFenced", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityActivationFenced = bool(v != 0) + case 30: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityFrontier", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + if err := m.DDLVisibilityFrontier.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -17749,6 +18908,26 @@ func (m *LogStoreHeartbeat) Unmarshal(dAtA []byte) error { } m.StoreIncarnation = string(dAtA[iNdEx:postIndex]) iNdEx = postIndex + case 12: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocolSupported", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityDeployedProtocolSupported = bool(v != 0) default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -23077,9 +24256,43 @@ func (m *CommandBatch) Unmarshal(dAtA []byte) error { break } } - case 4: + case 4: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field CommandIDs", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.CommandIDs = append(m.CommandIDs, ScheduleCommandID{}) + if err := m.CommandIDs[len(m.CommandIDs)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex + case 5: if wireType != 2 { - return fmt.Errorf("proto: wrong wireType = %d for field CommandIDs", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field ViewMetadataAdmission", wireType) } var msglen int for shift := uint(0); ; shift += 7 { @@ -23106,14 +24319,35 @@ func (m *CommandBatch) Unmarshal(dAtA []byte) error { if postIndex > l { return io.ErrUnexpectedEOF } - m.CommandIDs = append(m.CommandIDs, ScheduleCommandID{}) - if err := m.CommandIDs[len(m.CommandIDs)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + if m.ViewMetadataAdmission == nil { + m.ViewMetadataAdmission = &ViewMetadataAdmission{} + } + if err := m.ViewMetadataAdmission.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex - case 5: + case 6: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) + } + m.DDLVisibilityDeployedProtocol = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 7: if wireType != 2 { - return fmt.Errorf("proto: wrong wireType = %d for field ViewMetadataAdmission", wireType) + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityFrontier", wireType) } var msglen int for shift := uint(0); ; shift += 7 { @@ -23140,10 +24374,10 @@ func (m *CommandBatch) Unmarshal(dAtA []byte) error { if postIndex > l { return io.ErrUnexpectedEOF } - if m.ViewMetadataAdmission == nil { - m.ViewMetadataAdmission = &ViewMetadataAdmission{} + if m.DDLVisibilityFrontier == nil { + m.DDLVisibilityFrontier = ×tamp.Timestamp{} } - if err := m.ViewMetadataAdmission.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + if err := m.DDLVisibilityFrontier.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { return err } iNdEx = postIndex @@ -24249,6 +25483,118 @@ func (m *CNStoreInfo) Unmarshal(dAtA []byte) error { } } m.ViewMetadataIngressReady = bool(v != 0) + case 27: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityBarrierReady", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityBarrierReady = bool(v != 0) + case 28: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) + } + m.DDLVisibilityDeployedProtocol = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 29: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationPrepared", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityActivationPrepared = bool(v != 0) + case 30: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationFenced", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityActivationFenced = bool(v != 0) + case 31: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityFrontier", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + if err := m.DDLVisibilityFrontier.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -24429,6 +25775,58 @@ func (m *CNState) Unmarshal(dAtA []byte) error { } m.Stores[mapkey] = *mapvalue iNdEx = postIndex + case 2: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) + } + m.DDLVisibilityDeployedProtocol = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 3: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityFrontier", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + if err := m.DDLVisibilityFrontier.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -25882,6 +27280,58 @@ func (m *ClusterDetails) Unmarshal(dAtA []byte) error { return err } iNdEx = postIndex + case 7: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) + } + m.DDLVisibilityDeployedProtocol = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 8: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityFrontier", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthLogservice + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthLogservice + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + if err := m.DDLVisibilityFrontier.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) @@ -26952,6 +28402,26 @@ func (m *LogStoreInfo) Unmarshal(dAtA []byte) error { } m.StoreIncarnation = string(dAtA[iNdEx:postIndex]) iNdEx = postIndex + case 12: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocolSupported", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowLogservice + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityDeployedProtocolSupported = bool(v != 0) default: iNdEx = preIndex skippy, err := skipLogservice(dAtA[iNdEx:]) diff --git a/pkg/pb/logservice/logservice_test.go b/pkg/pb/logservice/logservice_test.go index b5db8b5a14a0d..e482c61a63f66 100644 --- a/pkg/pb/logservice/logservice_test.go +++ b/pkg/pb/logservice/logservice_test.go @@ -18,6 +18,7 @@ import ( "testing" "github.com/matrixorigin/matrixone/pkg/pb/metadata" + "github.com/matrixorigin/matrixone/pkg/pb/timestamp" "github.com/stretchr/testify/assert" ) @@ -35,23 +36,41 @@ func TestCNStateUpdate(t *testing.T) { state := CNState{Stores: map[string]CNStoreInfo{}} hb1 := CNStoreHeartbeat{ - UUID: "cn-a", - ServiceAddress: "addr-a", - Role: metadata.CNRole_AP, - CommandDeliveryAckSupported: true, + UUID: "cn-a", + ServiceAddress: "addr-a", + QueryAddress: "query-a", + Role: metadata.CNRole_AP, + CommandDeliveryAckSupported: true, + DDLVisibilityBarrierReady: true, + DDLVisibilityActivationPrepared: true, + DDLVisibilityActivationFenced: true, + DDLVisibilityDeployedProtocol: 38, + ViewMetadataAdmissionGeneration: 1, + ViewMetadataIngressReady: true, + DDLVisibilityEpochCommitTargets: []DDLVisibilityActivationTarget{{ + ServiceID: "cn-a", Generation: 1, QueryAddress: "query-a", + }}, } tick1 := uint64(100) state.Update(hb1, tick1) assert.Equal(t, state.Stores[hb1.UUID], CNStoreInfo{ - Tick: tick1, - ServiceAddress: hb1.ServiceAddress, - Role: metadata.CNRole_AP, - WorkState: metadata.WorkState_Working, - Labels: map[string]metadata.LabelList{}, - UpTime: state.Stores[hb1.UUID].UpTime, - CommandDeliveryAckSupported: true, + Tick: tick1, + ServiceAddress: hb1.ServiceAddress, + QueryAddress: hb1.QueryAddress, + Role: metadata.CNRole_AP, + WorkState: metadata.WorkState_Working, + Labels: map[string]metadata.LabelList{}, + UpTime: state.Stores[hb1.UUID].UpTime, + CommandDeliveryAckSupported: true, + DDLVisibilityBarrierReady: true, + DDLVisibilityActivationPrepared: true, + DDLVisibilityActivationFenced: true, + DDLVisibilityDeployedProtocol: 38, + ViewMetadataAdmissionGeneration: 1, + ViewMetadataIngressReady: true, }) + assert.Equal(t, int64(38), state.DDLVisibilityDeployedProtocol) hb2 := CNStoreHeartbeat{UUID: "cn-b", ServiceAddress: "addr-b", Role: metadata.CNRole_TP} tick2 := uint64(200) @@ -70,6 +89,8 @@ func TestCNStateUpdate(t *testing.T) { tick3 := uint64(300) state.Update(hb3, tick3) + assert.Equal(t, int64(38), state.DDLVisibilityDeployedProtocol, + "markerless/restarted heartbeats cannot lower the committed cluster epoch") assert.Equal(t, state.Stores[hb3.UUID], CNStoreInfo{ Tick: tick3, ServiceAddress: hb3.ServiceAddress, @@ -154,11 +175,12 @@ func TestLogStateUpdateStores(t *testing.T) { } hb1 := LogStoreHeartbeat{ - UUID: "log-a", - RaftAddress: "raft-a", - ServiceAddress: "addr-a", - GossipAddress: "gossip-a", - CommandDeliverySupported: true, + UUID: "log-a", + RaftAddress: "raft-a", + ServiceAddress: "addr-a", + GossipAddress: "gossip-a", + CommandDeliverySupported: true, + DDLVisibilityDeployedProtocolSupported: true, Replicas: []LogReplicaInfo{{ LogShardInfo: LogShardInfo{ ShardID: 1, @@ -173,12 +195,13 @@ func TestLogStateUpdateStores(t *testing.T) { tick1 := uint64(100) state.Update(hb1, tick1) assert.Equal(t, state.Stores[hb1.UUID], LogStoreInfo{ - Tick: tick1, - RaftAddress: hb1.RaftAddress, - ServiceAddress: hb1.ServiceAddress, - GossipAddress: hb1.GossipAddress, - Replicas: hb1.Replicas, - CommandDeliverySupported: true, + Tick: tick1, + RaftAddress: hb1.RaftAddress, + ServiceAddress: hb1.ServiceAddress, + GossipAddress: hb1.GossipAddress, + Replicas: hb1.Replicas, + CommandDeliverySupported: true, + DDLVisibilityDeployedProtocolSupported: true, }) hb2 := LogStoreHeartbeat{ @@ -593,6 +616,76 @@ func TestCNWorkStateUpdate(t *testing.T) { }) } +func TestCNStateAtomicallyRejectsEpochCommitAfterMarkerlessJoin(t *testing.T) { + state := NewCNState() + state.Update(CNStoreHeartbeat{ + UUID: "target-cn", QueryAddress: "target:6001", ViewMetadataAdmissionGeneration: 1, + DDLVisibilityBarrierReady: true, DDLVisibilityActivationPrepared: true, + DDLVisibilityActivationFenced: true, + }, 1) + commitTargets := []DDLVisibilityActivationTarget{{ + ServiceID: "target-cn", Generation: 1, QueryAddress: "target:6001", + }} + + // This join lands after the CN-side final scan but before the replicated + // epoch commit. The same RSM transition must see it and reject the stale set. + state.Update(CNStoreHeartbeat{ + UUID: "joining-cn", QueryAddress: "joining:6001", ViewMetadataAdmissionGeneration: 2, + DDLVisibilityBarrierReady: true, ViewMetadataIngressReady: true, + }, 2) + state.Update(CNStoreHeartbeat{ + UUID: "target-cn", QueryAddress: "target:6001", ViewMetadataAdmissionGeneration: 1, + DDLVisibilityBarrierReady: true, DDLVisibilityDeployedProtocol: 41, + DDLVisibilityEpochCommitTargets: commitTargets, + }, 3) + + assert.Equal(t, int64(0), state.DDLVisibilityDeployedProtocol) + assert.True(t, state.Stores["joining-cn"].ViewMetadataIngressReady) +} + +func TestCNStateAtomicallyRejectsEpochCommitFromUnfencedReplacement(t *testing.T) { + state := NewCNState() + oldFrontier := timestamp.Timestamp{PhysicalTime: 300} + state.Update(CNStoreHeartbeat{ + UUID: "target-cn", QueryAddress: "old:6001", ViewMetadataAdmissionGeneration: 1, + DDLVisibilityBarrierReady: true, DDLVisibilityActivationPrepared: true, + DDLVisibilityActivationFenced: true, DDLVisibilityFrontier: oldFrontier, + }, 1) + + // Replacement occurs after the old generation supplied Fenced. Even if the + // CN-side revalidation observes the new tuple, that tuple has no phase proof. + state.Update(CNStoreHeartbeat{ + UUID: "target-cn", QueryAddress: "new:6001", ViewMetadataAdmissionGeneration: 2, + DDLVisibilityBarrierReady: true, ViewMetadataIngressReady: true, + }, 2) + state.Update(CNStoreHeartbeat{ + UUID: "target-cn", QueryAddress: "new:6001", ViewMetadataAdmissionGeneration: 2, + DDLVisibilityBarrierReady: true, ViewMetadataIngressReady: true, + DDLVisibilityDeployedProtocol: 41, + DDLVisibilityEpochCommitTargets: []DDLVisibilityActivationTarget{{ + ServiceID: "target-cn", Generation: 2, QueryAddress: "new:6001", + }}, + }, 3) + + assert.Equal(t, int64(0), state.DDLVisibilityDeployedProtocol) + assert.True(t, state.Stores["target-cn"].ViewMetadataIngressReady) + assert.Equal(t, oldFrontier, state.DDLVisibilityFrontier, + "replacement heartbeat must not erase the cluster-lifetime DDL frontier") + delete(state.Stores, "target-cn") + assert.Equal(t, oldFrontier, state.DDLVisibilityFrontier, + "store removal must not erase the cluster-lifetime DDL frontier") +} + +func TestCNStateRejectsMarkerlessIngressAfterCommittedDDLCut(t *testing.T) { + state := NewCNState() + state.DDLVisibilityDeployedProtocol = 41 + state.Update(CNStoreHeartbeat{ + UUID: "markerless-cn", ViewMetadataIngressReady: true, + }, 1) + assert.False(t, state.Stores["markerless-cn"].ViewMetadataIngressReady) + assert.Equal(t, int64(41), state.DDLVisibilityDeployedProtocol) +} + func TestCNStateLabelPatch(t *testing.T) { state := CNState{Stores: map[string]CNStoreInfo{}} diff --git a/pkg/pb/metadata/metadata.pb.go b/pkg/pb/metadata/metadata.pb.go index c20c8defc9566..5c82f070292ab 100644 --- a/pkg/pb/metadata/metadata.pb.go +++ b/pkg/pb/metadata/metadata.pb.go @@ -497,10 +497,13 @@ type CNStore struct { // UUID CNStore uuid id UUID string `protobuf:"bytes,1,opt,name=UUID,proto3" json:"UUID,omitempty"` // Role CN role - Role CNRole `protobuf:"varint,2,opt,name=Role,proto3,enum=metadata.CNRole" json:"Role,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + Role CNRole `protobuf:"varint,2,opt,name=Role,proto3,enum=metadata.CNRole" json:"Role,omitempty"` + // DDLVisibilityDeployedProtocol is the durable activation level restored + // before this CN admits SQL after an ordinary restart. + DDLVisibilityDeployedProtocol int64 `protobuf:"varint,3,opt,name=DDLVisibilityDeployedProtocol,proto3" json:"DDLVisibilityDeployedProtocol,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CNStore) Reset() { *m = CNStore{} } @@ -550,6 +553,13 @@ func (m *CNStore) GetRole() CNRole { return CNRole_TP } +func (m *CNStore) GetDDLVisibilityDeployedProtocol() int64 { + if m != nil { + return m.DDLVisibilityDeployedProtocol + } + return 0 +} + // CNService cn service metadata type CNService struct { // ServiceID service ID @@ -572,13 +582,19 @@ type CNService struct { // CPUTotal is cpu quota from logservicepb.CNStore CPUTotal uint64 `protobuf:"varint,11,opt,name=CPUTotal,proto3" json:"CPUTotal,omitempty"` // MemTotal is mem quota from logservicepb.CNStore - MemTotal uint64 `protobuf:"varint,12,opt,name=MemTotal,proto3" json:"MemTotal,omitempty"` - ViewMetadataAdmissionGeneration uint64 `protobuf:"varint,13,opt,name=ViewMetadataAdmissionGeneration,proto3" json:"ViewMetadataAdmissionGeneration,omitempty"` - ViewMetadataAdmissionReady bool `protobuf:"varint,14,opt,name=ViewMetadataAdmissionReady,proto3" json:"ViewMetadataAdmissionReady,omitempty"` - ViewMetadataObservedEpoch uint64 `protobuf:"varint,15,opt,name=ViewMetadataObservedEpoch,proto3" json:"ViewMetadataObservedEpoch,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + MemTotal uint64 `protobuf:"varint,12,opt,name=MemTotal,proto3" json:"MemTotal,omitempty"` + ViewMetadataAdmissionGeneration uint64 `protobuf:"varint,13,opt,name=ViewMetadataAdmissionGeneration,proto3" json:"ViewMetadataAdmissionGeneration,omitempty"` + ViewMetadataAdmissionReady bool `protobuf:"varint,14,opt,name=ViewMetadataAdmissionReady,proto3" json:"ViewMetadataAdmissionReady,omitempty"` + ViewMetadataObservedEpoch uint64 `protobuf:"varint,15,opt,name=ViewMetadataObservedEpoch,proto3" json:"ViewMetadataObservedEpoch,omitempty"` + // DDLVisibilityBarrierReady is published after QueryService is listening. + // A CN is not publicly routable until its startup DDL frontier fence completes. + DDLVisibilityBarrierReady bool `protobuf:"varint,16,opt,name=DDLVisibilityBarrierReady,proto3" json:"DDLVisibilityBarrierReady,omitempty"` + // ViewMetadataIngressReady is retained in raw control-plane inventory so + // protocol activation can verify withdrawal/publication without routing it. + ViewMetadataIngressReady bool `protobuf:"varint,17,opt,name=ViewMetadataIngressReady,proto3" json:"ViewMetadataIngressReady,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *CNService) Reset() { *m = CNService{} } @@ -712,6 +728,20 @@ func (m *CNService) GetViewMetadataObservedEpoch() uint64 { return 0 } +func (m *CNService) GetDDLVisibilityBarrierReady() bool { + if m != nil { + return m.DDLVisibilityBarrierReady + } + return false +} + +func (m *CNService) GetViewMetadataIngressReady() bool { + if m != nil { + return m.ViewMetadataIngressReady + } + return false +} + // TNService tn service metadata type TNService struct { // ServiceID service ID @@ -902,65 +932,69 @@ func init() { func init() { proto.RegisterFile("metadata.proto", fileDescriptor_56d9f74966f40d04) } var fileDescriptor_56d9f74966f40d04 = []byte{ - // 918 bytes of a gzipped FileDescriptorProto - 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x56, 0xc1, 0x6e, 0xdb, 0x46, - 0x10, 0x35, 0x25, 0x5a, 0x22, 0x47, 0x8e, 0xca, 0x6c, 0xd0, 0x94, 0x75, 0x03, 0xd9, 0x60, 0x7b, - 0x70, 0x8d, 0x56, 0x4a, 0xdc, 0xa2, 0x28, 0x8a, 0xb6, 0x88, 0x2c, 0x27, 0x8e, 0x02, 0x85, 0x52, - 0x28, 0xca, 0x6d, 0x7a, 0x29, 0x28, 0x72, 0x43, 0x2f, 0x24, 0xed, 0x12, 0x2b, 0xca, 0x89, 0x7e, - 0xa1, 0xc7, 0x5e, 0x7b, 0xe9, 0xe7, 0xe4, 0xe8, 0x2f, 0x08, 0x5a, 0xf7, 0xd8, 0x9f, 0x08, 0xb8, - 0x24, 0x25, 0x9a, 0x96, 0x1c, 0x5f, 0x72, 0xe2, 0xce, 0xbc, 0xb7, 0xb3, 0xf3, 0x66, 0x67, 0x16, - 0x84, 0xea, 0x04, 0x87, 0x8e, 0xe7, 0x84, 0x4e, 0x3d, 0xe0, 0x2c, 0x64, 0x48, 0x49, 0xed, 0xed, - 0xaf, 0x7d, 0x12, 0x9e, 0xce, 0x86, 0x75, 0x97, 0x4d, 0x1a, 0x3e, 0xf3, 0x59, 0x43, 0x10, 0x86, - 0xb3, 0x97, 0xc2, 0x12, 0x86, 0x58, 0xc5, 0x1b, 0x8d, 0x36, 0xdc, 0xb2, 0xcd, 0xfe, 0xa9, 0xc3, - 0x3d, 0x0b, 0xbb, 0x8c, 0x7b, 0x48, 0x87, 0xb2, 0x30, 0xdb, 0x47, 0xba, 0xb4, 0x2b, 0xed, 0xc9, - 0x56, 0x6a, 0xa2, 0x1a, 0x40, 0x87, 0xf9, 0x29, 0x58, 0x10, 0x60, 0xc6, 0x63, 0xfc, 0x21, 0x41, - 0x39, 0x89, 0x85, 0x8e, 0x73, 0x61, 0x45, 0xac, 0xca, 0xc1, 0x27, 0xf5, 0x45, 0xde, 0x97, 0xe0, - 0x43, 0xe5, 0xcd, 0xdb, 0x9d, 0x8d, 0xf3, 0xb7, 0x3b, 0x92, 0x95, 0x4b, 0xe7, 0x1e, 0xa8, 0x16, - 0x0e, 0xc6, 0xc4, 0x75, 0x16, 0x67, 0x2e, 0x1d, 0x51, 0xb2, 0x4d, 0xcf, 0xe3, 0x78, 0x3a, 0xd5, - 0x8b, 0xbb, 0xd2, 0x9e, 0x6a, 0xa5, 0xa6, 0x71, 0x02, 0xd5, 0x34, 0xb5, 0xf7, 0x0a, 0xdb, 0x07, - 0xcd, 0x9c, 0x4d, 0x86, 0x98, 0x77, 0x5f, 0x26, 0xa1, 0xa7, 0xc9, 0x51, 0x57, 0xfc, 0xc6, 0x9f, - 0x12, 0x28, 0x69, 0x60, 0xf4, 0x34, 0x7f, 0x48, 0x22, 0x53, 0x5f, 0xca, 0xbc, 0x8c, 0x67, 0x74, - 0xe6, 0xd3, 0xbb, 0x5e, 0xe8, 0x3d, 0x50, 0x4d, 0x46, 0x4f, 0x58, 0x48, 0xa8, 0x2f, 0xa4, 0x2a, - 0xd6, 0xd2, 0x61, 0x98, 0xa2, 0xf0, 0x21, 0xe3, 0x18, 0x21, 0x90, 0x07, 0x83, 0x44, 0xa2, 0x6a, - 0x89, 0x35, 0x6a, 0x40, 0x49, 0x9c, 0x14, 0xa9, 0x2a, 0xee, 0x55, 0x0e, 0x6e, 0x5f, 0xb9, 0x85, - 0x43, 0x39, 0xca, 0xcb, 0x4a, 0x68, 0x06, 0x8f, 0x35, 0xae, 0x0d, 0x78, 0x3f, 0x17, 0x10, 0x5d, - 0xd5, 0x7b, 0x39, 0x22, 0xda, 0x85, 0x4a, 0x9b, 0xba, 0x0e, 0xa7, 0x4e, 0x48, 0x18, 0x4d, 0x2e, - 0x2b, 0xeb, 0x32, 0x5a, 0x50, 0x6e, 0x5d, 0xa3, 0xe1, 0x0b, 0x90, 0x2d, 0x36, 0xc6, 0xa2, 0x32, - 0xd5, 0x03, 0x6d, 0x79, 0x60, 0xcb, 0x8c, 0xfc, 0x96, 0x40, 0x8d, 0xff, 0x37, 0x41, 0x6d, 0x99, - 0x7d, 0xcc, 0xcf, 0x88, 0x8b, 0xa3, 0xa2, 0x25, 0xcb, 0x45, 0xb0, 0xa5, 0x03, 0xd5, 0x01, 0x75, - 0x98, 0x3b, 0x4a, 0x1c, 0x69, 0x1b, 0x15, 0x04, 0x6d, 0x05, 0x82, 0xbe, 0x83, 0xbb, 0x3d, 0x12, - 0xe0, 0x31, 0xa1, 0x38, 0xb7, 0x27, 0x56, 0xb3, 0x06, 0x8d, 0xc6, 0xa6, 0xff, 0xbc, 0x93, 0x72, - 0x65, 0xc1, 0xcd, 0x78, 0xd0, 0x4f, 0x50, 0xea, 0x38, 0x43, 0x3c, 0x9e, 0xea, 0x25, 0x51, 0xcc, - 0x9d, 0xac, 0xb6, 0x24, 0x56, 0x3d, 0x66, 0x3c, 0xa2, 0x21, 0x9f, 0xa7, 0x95, 0x8d, 0x5d, 0xe8, - 0x01, 0xa8, 0xbf, 0x30, 0x3e, 0xea, 0x87, 0x4e, 0x88, 0xf5, 0xb2, 0xa8, 0xce, 0x9d, 0x65, 0x84, - 0x05, 0x64, 0x2d, 0x59, 0xc8, 0x80, 0xad, 0xe7, 0x33, 0xcc, 0xe7, 0x69, 0x4e, 0x8a, 0xc8, 0xe9, - 0x92, 0x0f, 0xdd, 0x87, 0x3b, 0xe2, 0xea, 0x72, 0x52, 0x55, 0x41, 0x5d, 0x05, 0xa1, 0x6d, 0x50, - 0x5a, 0x6c, 0x32, 0x21, 0x61, 0xfb, 0x48, 0x07, 0x41, 0x5b, 0xd8, 0x02, 0xeb, 0x0d, 0x6c, 0x16, - 0x3a, 0x63, 0xbd, 0x22, 0x7a, 0x7b, 0x61, 0x47, 0xd8, 0x33, 0x3c, 0x89, 0xb1, 0xad, 0x18, 0x4b, - 0x6d, 0xf4, 0x04, 0x76, 0x4e, 0x08, 0x7e, 0xf5, 0x2c, 0x91, 0xd3, 0xf4, 0x26, 0x64, 0x3a, 0x25, - 0x8c, 0x1e, 0x63, 0x8a, 0x79, 0xdc, 0x4a, 0xb7, 0xc4, 0x96, 0xf7, 0xd1, 0xd0, 0xcf, 0xb0, 0xbd, - 0x92, 0x62, 0x61, 0xc7, 0x9b, 0xeb, 0x55, 0x31, 0x51, 0xd7, 0x30, 0xd0, 0x8f, 0xf0, 0x69, 0x16, - 0xed, 0x0e, 0xa7, 0x98, 0x9f, 0x61, 0xef, 0x51, 0xc0, 0xdc, 0x53, 0xfd, 0x23, 0x91, 0xc3, 0x7a, - 0xc2, 0xb6, 0x09, 0x95, 0xcc, 0x0d, 0x22, 0x0d, 0x8a, 0x23, 0x3c, 0x4f, 0x5a, 0x32, 0x5a, 0xa2, - 0x2f, 0x61, 0xf3, 0xcc, 0x19, 0xcf, 0xe2, 0xfe, 0xae, 0x64, 0x6f, 0x50, 0xec, 0xeb, 0x90, 0x69, - 0x68, 0xc5, 0x8c, 0x1f, 0x0a, 0xdf, 0x4b, 0x4f, 0x65, 0x65, 0x53, 0x2b, 0x19, 0x7f, 0xc9, 0xa0, - 0xda, 0x37, 0xec, 0xf6, 0xaf, 0xe0, 0xb6, 0xfd, 0x9a, 0xae, 0x6c, 0xf6, 0xab, 0x00, 0xfa, 0x16, - 0x3e, 0xee, 0x30, 0xdf, 0x76, 0xc8, 0x78, 0x65, 0xab, 0xaf, 0x06, 0xd7, 0x4c, 0x94, 0xbc, 0x76, - 0xa2, 0x96, 0xef, 0x52, 0xe9, 0x46, 0xef, 0x52, 0x66, 0x54, 0xca, 0xf9, 0x51, 0xb1, 0x6f, 0x30, - 0x2a, 0x1f, 0xa6, 0xef, 0x1f, 0xc2, 0x67, 0xcd, 0x59, 0xc8, 0xda, 0xd4, 0xe5, 0xe2, 0xb2, 0x1f, - 0x63, 0xea, 0xe2, 0xfe, 0x2c, 0x08, 0x18, 0x0f, 0xb1, 0x27, 0x46, 0x41, 0xb1, 0xae, 0xa3, 0x7c, - 0xa0, 0xee, 0xf8, 0x1c, 0xd4, 0x05, 0x8a, 0xee, 0x2e, 0x2a, 0x27, 0xed, 0x16, 0xf7, 0xd4, 0xb4, - 0x24, 0xfb, 0x4d, 0xa8, 0x24, 0x72, 0xec, 0x79, 0x80, 0x51, 0x09, 0x0a, 0x2d, 0x53, 0xdb, 0x88, - 0xbe, 0xb6, 0xa9, 0x49, 0xa8, 0x0c, 0xc5, 0x4e, 0xf7, 0x58, 0x2b, 0x20, 0x15, 0x36, 0x7b, 0x56, - 0xf7, 0xd7, 0x17, 0x5a, 0x11, 0x55, 0x01, 0x7a, 0x2f, 0xec, 0x27, 0x5d, 0xf3, 0xf7, 0xc1, 0xd1, - 0x63, 0x4d, 0xde, 0xd7, 0xa1, 0x14, 0xbf, 0xc1, 0x62, 0x57, 0x2f, 0xde, 0xdd, 0xec, 0x69, 0xd2, - 0xfe, 0xc3, 0xcc, 0xd3, 0x84, 0x2a, 0x50, 0x1e, 0xd0, 0x11, 0x65, 0xaf, 0xa8, 0xb6, 0x11, 0x19, - 0x11, 0x42, 0xa8, 0xaf, 0x49, 0x68, 0x0b, 0x94, 0x23, 0xee, 0x10, 0x1a, 0x59, 0x85, 0x08, 0x12, - 0x16, 0xf6, 0xb4, 0xe2, 0xe1, 0xf1, 0xf9, 0xbf, 0x35, 0xe9, 0xcd, 0x45, 0x4d, 0x3a, 0xbf, 0xa8, - 0x49, 0xff, 0x5c, 0xd4, 0x36, 0xfe, 0xfe, 0xaf, 0x26, 0xfd, 0xf6, 0x20, 0xf3, 0x8b, 0x33, 0x71, - 0x42, 0x4e, 0x5e, 0x33, 0x4e, 0x7c, 0x42, 0x53, 0x83, 0xe2, 0x46, 0x30, 0xf2, 0x1b, 0xc1, 0xb0, - 0x91, 0xd6, 0x69, 0x58, 0x12, 0x7f, 0x3b, 0xdf, 0xbc, 0x0b, 0x00, 0x00, 0xff, 0xff, 0xd8, 0x12, - 0x17, 0xdb, 0x38, 0x09, 0x00, 0x00, + // 980 bytes of a gzipped FileDescriptorProto + 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x56, 0xcd, 0x72, 0xda, 0x56, + 0x14, 0xb6, 0x00, 0x03, 0x3a, 0x38, 0x54, 0xbe, 0x99, 0xa6, 0xaa, 0x9b, 0x62, 0x8f, 0xda, 0x85, + 0xeb, 0x69, 0x21, 0x71, 0x3b, 0x9d, 0x4e, 0xa6, 0xed, 0x04, 0x43, 0xe2, 0x90, 0x21, 0x82, 0x08, + 0x70, 0x9b, 0x6e, 0x3a, 0x42, 0xba, 0x91, 0xef, 0x20, 0xee, 0xd5, 0x5c, 0x09, 0x27, 0x3c, 0x42, + 0xbb, 0xec, 0xb6, 0x9b, 0x3e, 0x4e, 0x96, 0x7e, 0x82, 0x4c, 0xeb, 0x4e, 0xdf, 0xa3, 0xa3, 0x2b, + 0x09, 0x04, 0x06, 0xe2, 0x4d, 0x56, 0xd6, 0x39, 0xdf, 0x77, 0x7e, 0x75, 0x3e, 0x19, 0x28, 0x8f, + 0x71, 0x60, 0xda, 0x66, 0x60, 0x56, 0x3d, 0xce, 0x02, 0x86, 0x8a, 0x89, 0xbd, 0xf7, 0x95, 0x43, + 0x82, 0xf3, 0xc9, 0xb0, 0x6a, 0xb1, 0x71, 0xcd, 0x61, 0x0e, 0xab, 0x09, 0xc2, 0x70, 0xf2, 0x52, + 0x58, 0xc2, 0x10, 0x4f, 0x51, 0xa0, 0xd6, 0x82, 0x5b, 0x7d, 0xbd, 0x77, 0x6e, 0x72, 0xdb, 0xc0, + 0x16, 0xe3, 0x36, 0x52, 0xa1, 0x20, 0xcc, 0x56, 0x53, 0x95, 0x0e, 0xa4, 0xc3, 0x9c, 0x91, 0x98, + 0xa8, 0x02, 0xd0, 0x66, 0x4e, 0x02, 0x66, 0x04, 0x98, 0xf2, 0x68, 0xbf, 0x4b, 0x50, 0x88, 0x73, + 0xa1, 0xd3, 0xa5, 0xb4, 0x22, 0x57, 0xe9, 0xf8, 0xa3, 0xea, 0xac, 0xef, 0x05, 0xf8, 0xa4, 0xf8, + 0xe6, 0xed, 0xfe, 0xd6, 0xe5, 0xdb, 0x7d, 0xc9, 0x58, 0x6a, 0xe7, 0x2e, 0xc8, 0x06, 0xf6, 0x5c, + 0x62, 0x99, 0xb3, 0x9a, 0x73, 0x47, 0xd8, 0x6c, 0xdd, 0xb6, 0x39, 0xf6, 0x7d, 0x35, 0x7b, 0x20, + 0x1d, 0xca, 0x46, 0x62, 0x6a, 0x67, 0x50, 0x4e, 0x5a, 0x7b, 0xe7, 0x60, 0x47, 0xa0, 0xe8, 0x93, + 0xf1, 0x10, 0xf3, 0xce, 0xcb, 0x38, 0xb5, 0x1f, 0x97, 0xba, 0xe6, 0xd7, 0xfe, 0x90, 0xa0, 0x98, + 0x24, 0x46, 0x4f, 0x97, 0x8b, 0xc4, 0x63, 0xaa, 0xf3, 0x31, 0x17, 0xf1, 0xd4, 0x9c, 0xcb, 0xed, + 0x6d, 0x1e, 0xf4, 0x2e, 0xc8, 0x3a, 0xa3, 0x67, 0x2c, 0x20, 0xd4, 0x11, 0xa3, 0x16, 0x8d, 0xb9, + 0x43, 0xd3, 0xc5, 0xe2, 0x03, 0xc6, 0x31, 0x42, 0x90, 0x1b, 0x0c, 0xe2, 0x11, 0x65, 0x43, 0x3c, + 0xa3, 0x1a, 0xe4, 0x45, 0xa5, 0x70, 0xaa, 0xec, 0x61, 0xe9, 0x78, 0xf7, 0xda, 0x5b, 0x38, 0xc9, + 0x85, 0x7d, 0x19, 0x31, 0x4d, 0xe3, 0xd1, 0x8c, 0x6b, 0x13, 0xde, 0x5b, 0x4a, 0x88, 0xae, 0xcf, + 0xbb, 0x98, 0x11, 0x1d, 0x40, 0xa9, 0x45, 0x2d, 0x93, 0x53, 0x33, 0x20, 0x8c, 0xc6, 0x2f, 0x2b, + 0xed, 0xd2, 0x7e, 0x93, 0xa0, 0xd0, 0xd8, 0x30, 0xc4, 0xe7, 0x90, 0x33, 0x98, 0x8b, 0xc5, 0x6a, + 0xca, 0xc7, 0xca, 0xbc, 0x62, 0x43, 0x0f, 0xfd, 0x86, 0x40, 0x51, 0x13, 0x3e, 0x6d, 0x36, 0xdb, + 0x67, 0xc4, 0x27, 0x43, 0xe2, 0x92, 0x60, 0xda, 0xc4, 0x9e, 0xcb, 0xa6, 0xd8, 0xee, 0x86, 0xc7, + 0x6e, 0x31, 0x57, 0x54, 0xce, 0x1a, 0x9b, 0x49, 0xda, 0x7f, 0x79, 0x90, 0x1b, 0x7a, 0x0f, 0xf3, + 0x0b, 0x62, 0xe1, 0x70, 0xf7, 0xf1, 0xe3, 0xac, 0xa5, 0xb9, 0x03, 0x55, 0x01, 0xb5, 0x99, 0x35, + 0x8a, 0x1d, 0xc9, 0x35, 0x66, 0x04, 0x6d, 0x05, 0x82, 0xbe, 0x85, 0x3b, 0x5d, 0xe2, 0x61, 0x97, + 0x50, 0xbc, 0x14, 0x13, 0x2d, 0x65, 0x0d, 0x1a, 0xaa, 0xaf, 0xf7, 0xbc, 0x9d, 0x70, 0x73, 0x82, + 0x9b, 0xf2, 0xa0, 0x1f, 0x20, 0xdf, 0x36, 0x87, 0xd8, 0xf5, 0xd5, 0xbc, 0x78, 0x27, 0xfb, 0xe9, + 0x0d, 0xc5, 0xb9, 0xaa, 0x11, 0xe3, 0x11, 0x0d, 0xf8, 0x34, 0x79, 0x41, 0x91, 0x0b, 0xdd, 0x07, + 0xf9, 0x27, 0xc6, 0x47, 0xbd, 0xc0, 0x0c, 0xb0, 0x5a, 0x10, 0x3b, 0xbe, 0x3d, 0xcf, 0x30, 0x83, + 0x8c, 0x39, 0x0b, 0x69, 0xb0, 0xf3, 0x7c, 0x82, 0xf9, 0x34, 0xe9, 0xa9, 0x28, 0x7a, 0x5a, 0xf0, + 0xa1, 0x7b, 0x70, 0x5b, 0x5c, 0xc0, 0xd2, 0xa8, 0xb2, 0xa0, 0xae, 0x82, 0xd0, 0x1e, 0x14, 0x1b, + 0x6c, 0x3c, 0x26, 0x41, 0xab, 0xa9, 0x82, 0xa0, 0xcd, 0x6c, 0x81, 0x75, 0x07, 0x7d, 0x16, 0x98, + 0xae, 0x5a, 0x12, 0x12, 0x99, 0xd9, 0x21, 0xf6, 0x0c, 0x8f, 0x23, 0x6c, 0x27, 0xc2, 0x12, 0x1b, + 0x3d, 0x81, 0xfd, 0x33, 0x82, 0x5f, 0x3d, 0x8b, 0xc7, 0xa9, 0xdb, 0x63, 0xe2, 0xfb, 0x84, 0xd1, + 0x53, 0x4c, 0x31, 0x8f, 0x2e, 0xf2, 0x96, 0x08, 0x79, 0x17, 0x0d, 0xfd, 0x08, 0x7b, 0x2b, 0x29, + 0x06, 0x36, 0xed, 0xa9, 0x5a, 0x16, 0xc2, 0xdc, 0xc0, 0x40, 0xdf, 0xc3, 0xc7, 0x69, 0xb4, 0x33, + 0xf4, 0x31, 0xbf, 0xc0, 0xf6, 0x23, 0x8f, 0x59, 0xe7, 0xea, 0x07, 0xa2, 0x87, 0xf5, 0x84, 0x30, + 0x7a, 0xe1, 0x70, 0x4f, 0x4c, 0xce, 0x09, 0xe6, 0x51, 0x71, 0x45, 0x14, 0x5f, 0x4f, 0x40, 0x0f, + 0x40, 0x4d, 0xa7, 0x6e, 0x51, 0x27, 0x5c, 0x78, 0x14, 0xbc, 0x2b, 0x82, 0xd7, 0xe2, 0x7b, 0x3a, + 0x94, 0x52, 0xb7, 0x83, 0x14, 0xc8, 0x8e, 0xf0, 0x34, 0x16, 0x43, 0xf8, 0x88, 0xbe, 0x80, 0xed, + 0x0b, 0xd3, 0x9d, 0x44, 0xfa, 0x2c, 0xa5, 0x6f, 0x47, 0xc4, 0xb5, 0x89, 0x1f, 0x18, 0x11, 0xe3, + 0x41, 0xe6, 0x3b, 0xe9, 0x69, 0xae, 0xb8, 0xad, 0xe4, 0xb5, 0x3f, 0x73, 0x20, 0xf7, 0x6f, 0xa8, + 0xb3, 0x2f, 0x61, 0xb7, 0xff, 0x9a, 0xae, 0x94, 0xd9, 0x75, 0x00, 0x7d, 0x03, 0x1f, 0xb6, 0x99, + 0xd3, 0x37, 0x89, 0xbb, 0x52, 0x64, 0xab, 0xc1, 0x35, 0x5a, 0xce, 0xad, 0xd5, 0xf2, 0xfc, 0xc3, + 0x9a, 0xbf, 0xd1, 0x87, 0x35, 0x25, 0xd2, 0xc2, 0xb2, 0x48, 0xfb, 0x37, 0x10, 0xe9, 0xfb, 0x51, + 0xdc, 0x43, 0xf8, 0xa4, 0x3e, 0x09, 0x58, 0x8b, 0x5a, 0x5c, 0x9c, 0xd9, 0x63, 0x4c, 0x2d, 0xdc, + 0x9b, 0x78, 0x1e, 0xe3, 0x01, 0xb6, 0x85, 0x08, 0x8b, 0xc6, 0x26, 0xca, 0x7b, 0xba, 0x8e, 0xcf, + 0x40, 0x9e, 0xa1, 0xe8, 0xce, 0x6c, 0x73, 0xd2, 0x41, 0xf6, 0x50, 0x4e, 0x56, 0x72, 0x54, 0x87, + 0x52, 0x3c, 0x4e, 0x7f, 0xea, 0x61, 0x94, 0x87, 0x4c, 0x43, 0x57, 0xb6, 0xc2, 0xbf, 0x7d, 0x5d, + 0x91, 0x50, 0x01, 0xb2, 0xed, 0xce, 0xa9, 0x92, 0x41, 0x32, 0x6c, 0x77, 0x8d, 0xce, 0xcf, 0x2f, + 0x94, 0x2c, 0x2a, 0x03, 0x74, 0x5f, 0xf4, 0x9f, 0x74, 0xf4, 0x5f, 0x07, 0xcd, 0xc7, 0x4a, 0xee, + 0x48, 0x85, 0x7c, 0xf4, 0x3f, 0x44, 0x44, 0x75, 0xa3, 0xe8, 0x7a, 0x57, 0x91, 0x8e, 0x1e, 0xa6, + 0x3e, 0x8a, 0xa8, 0x04, 0x85, 0x01, 0x1d, 0x51, 0xf6, 0x8a, 0x2a, 0x5b, 0xa1, 0x11, 0x22, 0x84, + 0x3a, 0x8a, 0x84, 0x76, 0xa0, 0xd8, 0xe4, 0x26, 0xa1, 0xa1, 0x95, 0x09, 0x21, 0x61, 0x61, 0x5b, + 0xc9, 0x9e, 0x9c, 0x5e, 0xfe, 0x53, 0x91, 0xde, 0x5c, 0x55, 0xa4, 0xcb, 0xab, 0x8a, 0xf4, 0xf7, + 0x55, 0x65, 0xeb, 0xaf, 0x7f, 0x2b, 0xd2, 0x2f, 0xf7, 0x53, 0xbf, 0xd1, 0xc6, 0x66, 0xc0, 0xc9, + 0x6b, 0xc6, 0x89, 0x43, 0x68, 0x62, 0x50, 0x5c, 0xf3, 0x46, 0x4e, 0xcd, 0x1b, 0xd6, 0x92, 0x3d, + 0x0d, 0xf3, 0xe2, 0xe7, 0xda, 0xd7, 0xff, 0x07, 0x00, 0x00, 0xff, 0xff, 0x8a, 0xee, 0xfb, 0x2c, + 0xf9, 0x09, 0x00, 0x00, } func (m *TNShardRecord) Marshal() (dAtA []byte, err error) { @@ -1265,6 +1299,11 @@ func (m *CNStore) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.DDLVisibilityDeployedProtocol != 0 { + i = encodeVarintMetadata(dAtA, i, uint64(m.DDLVisibilityDeployedProtocol)) + i-- + dAtA[i] = 0x18 + } if m.Role != 0 { i = encodeVarintMetadata(dAtA, i, uint64(m.Role)) i-- @@ -1304,6 +1343,30 @@ func (m *CNService) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.ViewMetadataIngressReady { + i-- + if m.ViewMetadataIngressReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0x88 + } + if m.DDLVisibilityBarrierReady { + i-- + if m.DDLVisibilityBarrierReady { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0x80 + } if m.ViewMetadataObservedEpoch != 0 { i = encodeVarintMetadata(dAtA, i, uint64(m.ViewMetadataObservedEpoch)) i-- @@ -1717,6 +1780,9 @@ func (m *CNStore) ProtoSize() (n int) { if m.Role != 0 { n += 1 + sovMetadata(uint64(m.Role)) } + if m.DDLVisibilityDeployedProtocol != 0 { + n += 1 + sovMetadata(uint64(m.DDLVisibilityDeployedProtocol)) + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -1784,6 +1850,12 @@ func (m *CNService) ProtoSize() (n int) { if m.ViewMetadataObservedEpoch != 0 { n += 1 + sovMetadata(uint64(m.ViewMetadataObservedEpoch)) } + if m.DDLVisibilityBarrierReady { + n += 3 + } + if m.ViewMetadataIngressReady { + n += 3 + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -2650,6 +2722,25 @@ func (m *CNStore) Unmarshal(dAtA []byte) error { break } } + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityDeployedProtocol", wireType) + } + m.DDLVisibilityDeployedProtocol = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowMetadata + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityDeployedProtocol |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipMetadata(dAtA[iNdEx:]) @@ -3169,6 +3260,46 @@ func (m *CNService) Unmarshal(dAtA []byte) error { break } } + case 16: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityBarrierReady", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowMetadata + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityBarrierReady = bool(v != 0) + case 17: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field ViewMetadataIngressReady", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowMetadata + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.ViewMetadataIngressReady = bool(v != 0) default: iNdEx = preIndex skippy, err := skipMetadata(dAtA[iNdEx:]) diff --git a/pkg/pb/query/query.pb.go b/pkg/pb/query/query.pb.go index ae10d4f3c2aca..3e09c4cd895be 100644 --- a/pkg/pb/query/query.pb.go +++ b/pkg/pb/query/query.pb.go @@ -109,6 +109,9 @@ const ( CmdMethod_IcebergCacheInvalidate CmdMethod = 38 // MongoDBClientRetire drains obsolete MongoDB driver clients after catalog DDL. CmdMethod_MongoDBClientRetire CmdMethod = 39 + // SyncCommitV2 waits for local logtail application with the request context. + // Unlike legacy SyncCommit, cancellation is returned to the sender. + CmdMethod_SyncCommitV2 CmdMethod = 40 ) var CmdMethod_name = map[int32]string{ @@ -152,6 +155,7 @@ var CmdMethod_name = map[int32]string{ 37: "ISCPDrainConsumer", 38: "IcebergCacheInvalidate", 39: "MongoDBClientRetire", + 40: "SyncCommitV2", } var CmdMethod_value = map[string]int32{ @@ -195,6 +199,7 @@ var CmdMethod_value = map[string]int32{ "ISCPDrainConsumer": 37, "IcebergCacheInvalidate": 38, "MongoDBClientRetire": 39, + "SyncCommitV2": 40, } func (x CmdMethod) String() string { @@ -658,6 +663,12 @@ var xxx_messageInfo_GetProtocolVersionRequest proto.InternalMessageInfo type GetProtocolVersionResponse struct { // ProtocolVersion is the version of the protocol Version int64 `protobuf:"varint,1,opt,name=Version,proto3" json:"Version,omitempty"` + // DDLVisibilityActivationPrepared means this CN has blocked and drained DDL + // producers before exposing the v36 sender capability. + DDLVisibilityActivationPrepared bool `protobuf:"varint,2,opt,name=DDLVisibilityActivationPrepared,proto3" json:"DDLVisibilityActivationPrepared,omitempty"` + // DDLVisibilityActivationFenced means this CN applied the converged peer + // frontier while every participant's DDL producers were blocked. + DDLVisibilityActivationFenced bool `protobuf:"varint,3,opt,name=DDLVisibilityActivationFenced,proto3" json:"DDLVisibilityActivationFenced,omitempty"` } func (m *GetProtocolVersionResponse) Reset() { *m = GetProtocolVersionResponse{} } @@ -700,9 +711,29 @@ func (m *GetProtocolVersionResponse) GetVersion() int64 { return 0 } +func (m *GetProtocolVersionResponse) GetDDLVisibilityActivationPrepared() bool { + if m != nil { + return m.DDLVisibilityActivationPrepared + } + return false +} + +func (m *GetProtocolVersionResponse) GetDDLVisibilityActivationFenced() bool { + if m != nil { + return m.DDLVisibilityActivationFenced + } + return false +} + type SetProtocolVersionRequest struct { // ProtocolVersion is the version of the protocol Version int64 `protobuf:"varint,1,opt,name=Version,proto3" json:"Version,omitempty"` + // DDLVisibilityActivationTargets is the complete CN set participating in a + // coordinated live activation. It is empty for ordinary version updates. + DDLVisibilityActivationTargets []string `protobuf:"bytes,2,rep,name=DDLVisibilityActivationTargets,proto3" json:"DDLVisibilityActivationTargets,omitempty"` + // The authoritative raw-inventory identity selected by the control plane. + DDLVisibilityTargetGeneration uint64 `protobuf:"varint,3,opt,name=DDLVisibilityTargetGeneration,proto3" json:"DDLVisibilityTargetGeneration,omitempty"` + DDLVisibilityTargetQueryAddress string `protobuf:"bytes,4,opt,name=DDLVisibilityTargetQueryAddress,proto3" json:"DDLVisibilityTargetQueryAddress,omitempty"` } func (m *SetProtocolVersionRequest) Reset() { *m = SetProtocolVersionRequest{} } @@ -745,6 +776,27 @@ func (m *SetProtocolVersionRequest) GetVersion() int64 { return 0 } +func (m *SetProtocolVersionRequest) GetDDLVisibilityActivationTargets() []string { + if m != nil { + return m.DDLVisibilityActivationTargets + } + return nil +} + +func (m *SetProtocolVersionRequest) GetDDLVisibilityTargetGeneration() uint64 { + if m != nil { + return m.DDLVisibilityTargetGeneration + } + return 0 +} + +func (m *SetProtocolVersionRequest) GetDDLVisibilityTargetQueryAddress() string { + if m != nil { + return m.DDLVisibilityTargetQueryAddress + } + return "" +} + type SetProtocolVersionResponse struct { // ProtocolVersion is the version of the protocol Version int64 `protobuf:"varint,1,opt,name=Version,proto3" json:"Version,omitempty"` @@ -5974,286 +6026,292 @@ func init() { func init() { proto.RegisterFile("query.proto", fileDescriptor_5c6ac9b241082464) } var fileDescriptor_5c6ac9b241082464 = []byte{ - // 4455 bytes of a gzipped FileDescriptorProto - 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x5c, 0x5b, 0x73, 0xdb, 0x48, - 0x76, 0x36, 0xad, 0x1b, 0x79, 0x44, 0x49, 0x50, 0x5b, 0x17, 0x48, 0x96, 0x29, 0x19, 0xe3, 0xb1, - 0x65, 0x7b, 0x47, 0x9a, 0x78, 0xc7, 0x9e, 0xbd, 0x4c, 0x52, 0x23, 0x51, 0x96, 0x4d, 0x5b, 0xb2, - 0xe4, 0x26, 0x7d, 0xd9, 0xd9, 0xaa, 0xa9, 0x82, 0xc8, 0x16, 0x85, 0x15, 0x09, 0x70, 0x00, 0xd0, - 0x23, 0x4d, 0x55, 0xfe, 0x40, 0x1e, 0x52, 0xfb, 0x13, 0x92, 0xaa, 0x54, 0xe5, 0x2d, 0x8f, 0x79, - 0xdf, 0xb7, 0x7d, 0xdc, 0x87, 0x54, 0x65, 0xf3, 0x92, 0xa4, 0x66, 0xfe, 0x43, 0x5e, 0xb3, 0xd5, - 0x8d, 0x6e, 0x00, 0xdd, 0x68, 0x80, 0x9e, 0xcb, 0xbe, 0xb8, 0xd8, 0xa7, 0xcf, 0xf9, 0xba, 0xd1, - 0x97, 0xd3, 0xe7, 0xeb, 0xd3, 0x32, 0x4c, 0x7f, 0x35, 0x24, 0xfe, 0xe5, 0xd6, 0xc0, 0xf7, 0x42, - 0x0f, 0x4d, 0xb0, 0xc2, 0x6a, 0x35, 0x08, 0xed, 0x70, 0x18, 0x44, 0xc2, 0x55, 0xe8, 0x79, 0xed, - 0x73, 0xfe, 0xbb, 0x12, 0x5e, 0xb8, 0xfc, 0xe7, 0x5c, 0xe8, 0xf4, 0x49, 0x10, 0xda, 0xfd, 0x81, - 0x10, 0x50, 0xab, 0xc0, 0x71, 0x4f, 0x3d, 0x61, 0x38, 0xe8, 0xd9, 0x42, 0xfb, 0xa3, 0xae, 0x13, - 0x9e, 0x0d, 0x4f, 0xb6, 0xda, 0x5e, 0x7f, 0xbb, 0xeb, 0x75, 0xbd, 0x6d, 0x26, 0x3e, 0x19, 0x9e, - 0xb2, 0x12, 0x2b, 0xb0, 0x5f, 0x5c, 0x7d, 0xbd, 0xeb, 0x79, 0xdd, 0x1e, 0x49, 0xb4, 0x94, 0xc6, - 0xac, 0x5b, 0x50, 0x7d, 0x49, 0xfb, 0x8a, 0xc9, 0x57, 0x43, 0x12, 0x84, 0x68, 0x01, 0x26, 0x58, - 0xd9, 0x2c, 0x6d, 0x94, 0x36, 0x2b, 0x38, 0x2a, 0x58, 0x2f, 0x60, 0xa9, 0x79, 0xe6, 0x7d, 0x7d, - 0xec, 0x7b, 0x6d, 0x12, 0x04, 0x07, 0x4e, 0x10, 0x0a, 0xfd, 0x25, 0x98, 0x6c, 0x11, 0xd7, 0x76, - 0x43, 0x6e, 0xc0, 0x4b, 0x68, 0x0d, 0x2a, 0xcd, 0xcb, 0x80, 0x57, 0x5d, 0xdd, 0x28, 0x6d, 0x96, - 0x71, 0x22, 0xb0, 0xde, 0xc0, 0x7c, 0xf3, 0xd2, 0x6d, 0xd7, 0xbd, 0x7e, 0xdf, 0x89, 0xa1, 0x76, - 0x61, 0xf6, 0xc0, 0x0e, 0x49, 0x10, 0x46, 0xe2, 0x56, 0x93, 0x41, 0x4e, 0x3f, 0x58, 0xd8, 0x4a, - 0x3a, 0xdd, 0x12, 0xbf, 0x76, 0xc7, 0xff, 0xf8, 0xdf, 0xeb, 0x57, 0xb0, 0x62, 0x61, 0x7d, 0x01, - 0x28, 0x0d, 0x1c, 0x0c, 0x3c, 0x37, 0x20, 0x68, 0x0f, 0xe6, 0xea, 0x43, 0xdf, 0x27, 0xee, 0xf7, - 0x81, 0x56, 0x4d, 0x2c, 0x04, 0xc6, 0x13, 0x12, 0x4a, 0x7d, 0xb6, 0x7e, 0x03, 0xf3, 0x29, 0xd9, - 0x4f, 0xda, 0xdc, 0x36, 0x2c, 0xd6, 0x3d, 0x9f, 0xec, 0x0d, 0xfb, 0x83, 0xba, 0xe7, 0x9e, 0x3a, - 0xdd, 0xd4, 0x90, 0xef, 0xb4, 0x43, 0xc7, 0x73, 0xc5, 0x90, 0x47, 0x25, 0xcb, 0x84, 0x25, 0xd5, - 0x20, 0xea, 0x90, 0x75, 0x1d, 0x56, 0x9e, 0x90, 0xf0, 0x98, 0x4e, 0x78, 0xdb, 0xeb, 0xbd, 0x26, - 0x7e, 0xe0, 0x78, 0xae, 0xf8, 0x84, 0x47, 0xb0, 0xaa, 0xab, 0xe4, 0xdf, 0x62, 0xc2, 0x14, 0x17, - 0xb1, 0xd6, 0xc6, 0xb0, 0x28, 0x5a, 0x0f, 0x61, 0xa5, 0x99, 0x07, 0x5a, 0x60, 0xf6, 0x08, 0x56, - 0x9b, 0x3f, 0xa4, 0xb9, 0x9f, 0xc1, 0x2c, 0x1e, 0xba, 0x2d, 0x3b, 0x38, 0x17, 0x6d, 0xac, 0x42, - 0x99, 0x16, 0xeb, 0x5e, 0x87, 0x30, 0xe5, 0x09, 0x1c, 0x97, 0xad, 0xbb, 0x30, 0x17, 0x6b, 0x73, - 0xe8, 0x25, 0x98, 0xc4, 0x24, 0x18, 0xf6, 0xe2, 0x95, 0x1a, 0x95, 0xe8, 0xb0, 0xd1, 0xef, 0x77, - 0x06, 0xa4, 0xe7, 0xb8, 0xa4, 0xe1, 0x9e, 0x7a, 0x62, 0x64, 0xb6, 0x61, 0x39, 0x53, 0xc3, 0xc1, - 0x16, 0x60, 0xa2, 0xee, 0x0d, 0xf9, 0xaa, 0x1f, 0xc3, 0x51, 0xc1, 0xfa, 0xf7, 0x15, 0x98, 0x12, - 0xbd, 0x5b, 0x83, 0x0a, 0xff, 0xd9, 0xd8, 0x63, 0x5a, 0xe3, 0x38, 0x11, 0xa0, 0x2d, 0xa8, 0xd4, - 0xfb, 0x9d, 0x43, 0x12, 0x9e, 0x79, 0x1d, 0xb6, 0x3d, 0x66, 0x1f, 0x18, 0x5b, 0x91, 0x07, 0x89, - 0xe5, 0x38, 0x51, 0x41, 0x9f, 0xca, 0xdb, 0xd4, 0x1c, 0x63, 0xeb, 0xe9, 0x1a, 0x37, 0x49, 0x57, - 0x61, 0x79, 0x3f, 0xbf, 0xca, 0xdb, 0xb9, 0xe6, 0x38, 0x83, 0xb8, 0xc1, 0x21, 0xf4, 0x4a, 0x38, - 0x6f, 0xdb, 0x1f, 0xc0, 0xb5, 0x9d, 0x5e, 0x48, 0xfc, 0x9d, 0x76, 0x9b, 0x7e, 0xb9, 0xc0, 0x9c, - 0x60, 0x98, 0xab, 0x1c, 0x53, 0xa3, 0x81, 0x75, 0x66, 0xe8, 0x73, 0x98, 0x7b, 0xee, 0xf4, 0x7a, - 0x75, 0xcf, 0x15, 0x0b, 0xc8, 0x9c, 0x64, 0x48, 0x4b, 0x1c, 0x49, 0xa9, 0xc5, 0xaa, 0x3a, 0xaa, - 0x83, 0xd1, 0xf2, 0xed, 0x36, 0x69, 0x0e, 0xec, 0x18, 0x62, 0x8a, 0x41, 0x2c, 0x73, 0x08, 0xb5, - 0x1a, 0x67, 0x0c, 0x50, 0x03, 0xd0, 0x13, 0x12, 0x1e, 0x78, 0xed, 0xf3, 0xd4, 0x2a, 0x30, 0xcb, - 0x0c, 0x66, 0x85, 0xc3, 0x64, 0x15, 0xb0, 0xc6, 0x08, 0xed, 0x33, 0xbf, 0xd0, 0xba, 0x70, 0xd3, - 0x48, 0x15, 0x86, 0x64, 0x26, 0x48, 0x72, 0x3d, 0xce, 0x9a, 0xd0, 0x71, 0xa6, 0xfe, 0xc5, 0x6e, - 0x9f, 0xa5, 0x57, 0xa6, 0x09, 0xd2, 0x38, 0x6b, 0x34, 0xb0, 0xce, 0x0c, 0xfd, 0x02, 0xa0, 0x79, - 0xd9, 0x76, 0x23, 0x17, 0x63, 0x4e, 0x4b, 0xdd, 0xc9, 0xf8, 0x63, 0x9c, 0xd2, 0x45, 0x0f, 0xa1, - 0x12, 0xfb, 0x39, 0xb3, 0x2a, 0x0d, 0xac, 0xea, 0x13, 0x71, 0xa2, 0x89, 0x8e, 0xd9, 0x88, 0x2a, - 0x9b, 0xdd, 0x9c, 0x61, 0xf6, 0x1b, 0x89, 0xbd, 0xde, 0x89, 0x60, 0x8d, 0x2d, 0x45, 0xcc, 0xba, - 0x0f, 0x73, 0x56, 0x42, 0x6c, 0xe6, 0x23, 0x66, 0xab, 0xd0, 0x1e, 0xcc, 0xca, 0x6e, 0xd3, 0x9c, - 0x63, 0x68, 0x6b, 0x62, 0x3f, 0xea, 0x9c, 0x30, 0x56, 0x6c, 0xd0, 0x36, 0x4c, 0x71, 0x87, 0x63, - 0x1a, 0xcc, 0x7c, 0x91, 0x9b, 0xcb, 0x4e, 0x0b, 0x0b, 0x2d, 0xf4, 0x1b, 0x58, 0xc4, 0xa4, 0xef, - 0xbd, 0x23, 0xf4, 0xdf, 0x90, 0xd0, 0x05, 0xd4, 0xb2, 0x4f, 0x7a, 0xc4, 0x9c, 0x67, 0xe6, 0x1f, - 0x08, 0x73, 0x9d, 0x8e, 0x00, 0xd3, 0x23, 0xa0, 0x1d, 0x98, 0xa1, 0x4b, 0x92, 0x9d, 0x8c, 0xbb, - 0x8e, 0xdb, 0x31, 0x11, 0x83, 0xbc, 0x9e, 0x5a, 0xc2, 0x71, 0x9d, 0x80, 0x92, 0x2d, 0xd0, 0x33, - 0x30, 0x5e, 0xb9, 0xc1, 0xf0, 0x24, 0x68, 0xfb, 0xce, 0x09, 0x89, 0x3a, 0x76, 0x8d, 0xa1, 0xd4, - 0x38, 0x8a, 0x5a, 0x1d, 0x6f, 0x2b, 0xb5, 0x22, 0xbd, 0x86, 0xf7, 0xec, 0xd0, 0x16, 0x6b, 0x78, - 0x41, 0xbb, 0x86, 0x53, 0x1a, 0x58, 0x67, 0xc6, 0xd1, 0x9a, 0x34, 0x44, 0x4a, 0xef, 0x88, 0x45, - 0x15, 0x4d, 0xd5, 0xc0, 0x3a, 0x33, 0xea, 0x1e, 0xf5, 0xce, 0xdf, 0x5c, 0x92, 0xdc, 0xa3, 0x5e, - 0x09, 0xe7, 0x18, 0x53, 0xd8, 0x43, 0xa7, 0xeb, 0xdb, 0x21, 0xa1, 0x4e, 0x6a, 0xdf, 0xf7, 0xfa, - 0x02, 0x76, 0x59, 0x82, 0xd5, 0x2b, 0xe1, 0x1c, 0x63, 0x74, 0x04, 0x0b, 0xa9, 0x9a, 0x56, 0xdc, - 0x57, 0x53, 0x9a, 0x5f, 0x9d, 0x0a, 0xd6, 0x1a, 0xa2, 0x13, 0x30, 0x31, 0xe9, 0x79, 0x76, 0x67, - 0x67, 0x18, 0x7a, 0x0d, 0xb7, 0xed, 0x93, 0x3e, 0x0d, 0x41, 0xe8, 0x98, 0x9b, 0x2b, 0x0c, 0xf4, - 0x76, 0xbc, 0x0e, 0xf5, 0x6a, 0x02, 0x3f, 0x17, 0x87, 0xba, 0xe6, 0x7a, 0xd8, 0xc3, 0xc4, 0xee, - 0x10, 0x5f, 0x74, 0x78, 0x55, 0xf2, 0x20, 0x6a, 0x35, 0xce, 0x18, 0xa0, 0x43, 0x98, 0x7b, 0x42, - 0x42, 0x4c, 0x06, 0x3d, 0xa7, 0x6d, 0x47, 0x27, 0xef, 0x75, 0x75, 0x82, 0xd2, 0xb5, 0xdc, 0x4e, - 0xc4, 0x56, 0x4a, 0x2d, 0x5d, 0x44, 0x98, 0x04, 0x24, 0x6c, 0x92, 0x20, 0xe5, 0x1e, 0xcc, 0x35, - 0x69, 0x11, 0x69, 0x34, 0xb0, 0xce, 0x0c, 0x1d, 0xc0, 0xfc, 0x13, 0xef, 0xd0, 0xbe, 0xa0, 0xe7, - 0x64, 0x20, 0xb0, 0x6e, 0xc8, 0xce, 0x5e, 0xad, 0xe7, 0x3d, 0xcb, 0x1a, 0x72, 0x34, 0xd2, 0x3f, - 0x70, 0x12, 0x9f, 0x6a, 0xd6, 0x54, 0x34, 0xb9, 0x3e, 0x85, 0x26, 0x57, 0xa0, 0x2f, 0x61, 0x79, - 0xdf, 0xe9, 0x91, 0x26, 0xf1, 0xdf, 0x39, 0x6d, 0x92, 0x9e, 0x32, 0x73, 0x5d, 0xda, 0xcf, 0x39, - 0x5a, 0x1c, 0x39, 0x0f, 0x04, 0xf5, 0x61, 0x4d, 0xad, 0x7a, 0xfc, 0xce, 0x69, 0xc7, 0x1d, 0xdf, - 0x90, 0xbc, 0x59, 0x91, 0x2a, 0x6f, 0xa9, 0x10, 0x0e, 0xbd, 0x82, 0x85, 0x43, 0x12, 0xda, 0x1d, - 0x3b, 0xb4, 0xa5, 0x6f, 0xb9, 0x29, 0xef, 0x00, 0x8d, 0x0a, 0x87, 0xd7, 0x9a, 0xa3, 0x23, 0x40, - 0x4f, 0xbc, 0x27, 0xf5, 0x63, 0xe2, 0xb7, 0x49, 0x12, 0xcd, 0x58, 0xf2, 0xc9, 0x9f, 0x51, 0xe0, - 0x90, 0x1a, 0x53, 0x1a, 0x4a, 0xec, 0xdb, 0xc3, 0x5e, 0xd8, 0x70, 0x7f, 0x47, 0x92, 0xc1, 0xf8, - 0x40, 0x02, 0xcc, 0x2a, 0x60, 0x8d, 0x11, 0xfa, 0x2d, 0x2c, 0xd5, 0xc3, 0xde, 0xa1, 0xc7, 0x9c, - 0x29, 0x73, 0x60, 0x02, 0xee, 0x96, 0xb4, 0x03, 0xf4, 0x4a, 0xbc, 0x8f, 0x39, 0x10, 0xe8, 0x4b, - 0x58, 0x79, 0xe3, 0xf9, 0xe7, 0xc1, 0xc0, 0x6e, 0x93, 0xd6, 0x99, 0x4f, 0x82, 0x33, 0xaf, 0x27, - 0xce, 0x04, 0xf3, 0x43, 0xe9, 0x54, 0xcd, 0xd5, 0xc3, 0xf9, 0x10, 0xa8, 0x0f, 0xb5, 0x7a, 0xd8, - 0x3b, 0xf6, 0xc9, 0x29, 0x09, 0xdb, 0x67, 0x47, 0x6e, 0x53, 0x1c, 0x0d, 0x71, 0x23, 0xb7, 0x59, - 0x23, 0x1f, 0x26, 0x1f, 0x51, 0xa0, 0x8c, 0x47, 0x80, 0xa1, 0xdf, 0x82, 0xd9, 0x68, 0xd6, 0x8f, - 0xf7, 0x7c, 0xdb, 0x71, 0xeb, 0x9e, 0x1b, 0x0c, 0xfb, 0x89, 0xcf, 0xb9, 0xc3, 0x1a, 0x5a, 0xe7, - 0x0d, 0xe5, 0xa9, 0xe1, 0x5c, 0x00, 0x34, 0x80, 0x1b, 0x8d, 0x36, 0x39, 0x21, 0x7e, 0x97, 0x07, - 0x56, 0xef, 0xec, 0x9e, 0xd3, 0xb1, 0xc3, 0x78, 0x11, 0x6e, 0xb2, 0x16, 0x6e, 0x89, 0x16, 0x8a, - 0x74, 0xf9, 0xb4, 0x14, 0x03, 0xa2, 0x2e, 0xac, 0x1e, 0x7a, 0x6e, 0xd7, 0xdb, 0xdb, 0xad, 0xf7, - 0x1c, 0xb6, 0xba, 0x42, 0xc7, 0x8f, 0x9b, 0xbb, 0xcb, 0x9a, 0xbb, 0x29, 0xd6, 0x7c, 0xae, 0x22, - 0x6f, 0xab, 0x00, 0xca, 0xda, 0x87, 0xe5, 0x4c, 0xa0, 0xcf, 0x99, 0xce, 0x7d, 0x28, 0x73, 0x77, - 0x17, 0x98, 0xa5, 0x8d, 0xb1, 0xcd, 0xe9, 0x07, 0x73, 0x5b, 0xfc, 0x5a, 0x43, 0xb8, 0xc1, 0x58, - 0xc1, 0xfa, 0xc3, 0x2a, 0x94, 0x63, 0xcb, 0x9f, 0x96, 0x01, 0x2d, 0xc0, 0xc4, 0x63, 0xdf, 0xf7, - 0x7c, 0x46, 0x7d, 0xaa, 0x38, 0x2a, 0xa0, 0xb7, 0xb9, 0x1d, 0xe7, 0xfc, 0xa6, 0x96, 0xc7, 0x6f, - 0x22, 0x2d, 0x9c, 0xfb, 0xdd, 0x47, 0xb0, 0x20, 0x53, 0x15, 0x0e, 0x3b, 0x21, 0x79, 0x1a, 0x9d, - 0x0a, 0xd6, 0x1a, 0xd2, 0x73, 0x30, 0x61, 0x2d, 0x1c, 0x6c, 0x52, 0x3a, 0x07, 0xd5, 0x6a, 0x9c, - 0x31, 0xa0, 0xbc, 0x22, 0x45, 0x5b, 0x38, 0xca, 0x94, 0x74, 0x38, 0x64, 0xea, 0x71, 0xd6, 0x84, - 0x47, 0x51, 0x09, 0x6b, 0xe1, 0x48, 0x65, 0x35, 0x8a, 0x52, 0x35, 0xb0, 0xce, 0x8c, 0x13, 0xa7, - 0x98, 0xba, 0x70, 0xb0, 0x8a, 0x4a, 0x9c, 0x14, 0x05, 0xac, 0x31, 0xa2, 0xc3, 0x2e, 0x33, 0x17, - 0x0e, 0x06, 0x6a, 0x08, 0x9b, 0x51, 0xc1, 0x5a, 0x43, 0xf4, 0x4b, 0xca, 0x79, 0x04, 0xb5, 0xe1, - 0x9c, 0x67, 0x45, 0xc3, 0x79, 0x38, 0x48, 0x4a, 0x19, 0x3d, 0xca, 0x92, 0x1e, 0x33, 0x4b, 0x7a, - 0xb8, 0x61, 0x8a, 0xf5, 0xbc, 0x2c, 0x60, 0x3d, 0x37, 0x0b, 0x58, 0x4f, 0x6a, 0x58, 0x54, 0x92, - 0xf2, 0xb2, 0x80, 0xf6, 0xdc, 0x2c, 0xa0, 0x3d, 0x02, 0x52, 0xc3, 0x7b, 0x1e, 0xe7, 0xf0, 0x9e, - 0x1b, 0x39, 0xbc, 0x87, 0x43, 0xa9, 0xc4, 0xe7, 0x63, 0x95, 0xf8, 0x2c, 0xa9, 0xc4, 0x87, 0x1b, - 0xc6, 0xcc, 0xe7, 0x8b, 0x62, 0xe6, 0x73, 0xab, 0x98, 0xf9, 0x70, 0xb4, 0x1c, 0xea, 0xb3, 0xab, - 0xa7, 0x3e, 0x6b, 0x7a, 0xea, 0xc3, 0xb1, 0x14, 0xee, 0xf3, 0x3c, 0x97, 0xfb, 0xac, 0xe7, 0x72, - 0x1f, 0xb1, 0x61, 0x33, 0xe4, 0x27, 0xb5, 0x9e, 0x23, 0x16, 0xc3, 0xd7, 0xf3, 0x82, 0x76, 0x3d, - 0xa7, 0x55, 0xb0, 0xd6, 0x90, 0x03, 0xa6, 0x88, 0x0c, 0x07, 0x5c, 0x54, 0x01, 0x33, 0x2a, 0x58, - 0x6b, 0x48, 0x5d, 0x68, 0xce, 0x2d, 0x17, 0xe7, 0x40, 0xb5, 0x3c, 0x0e, 0x24, 0x5c, 0x68, 0xde, - 0x25, 0xd9, 0x5b, 0x58, 0xce, 0x10, 0x19, 0x8e, 0xbc, 0x2c, 0x21, 0xe7, 0x68, 0xe1, 0x3c, 0x73, - 0x84, 0x61, 0x51, 0xe1, 0x33, 0x1c, 0xd7, 0x94, 0xa6, 0x5b, 0xab, 0x83, 0xf5, 0xa6, 0xa8, 0x3d, - 0x92, 0x0b, 0xdd, 0x19, 0xc9, 0x85, 0x78, 0x0b, 0xf9, 0x64, 0x68, 0x1f, 0xe6, 0x53, 0xdc, 0x86, - 0x77, 0x7a, 0x55, 0x72, 0x2d, 0x99, 0x7a, 0x9c, 0x35, 0x41, 0x2f, 0xf2, 0xf8, 0x50, 0x2d, 0x8f, - 0x0f, 0x45, 0x86, 0x79, 0x84, 0xe8, 0x08, 0x16, 0x64, 0x66, 0xc3, 0xbb, 0xb6, 0x26, 0xad, 0x2a, - 0x9d, 0x0a, 0xd6, 0x1a, 0x46, 0x11, 0x75, 0x42, 0x6d, 0x38, 0xdc, 0x0d, 0x25, 0xa2, 0x56, 0x15, - 0x92, 0x88, 0x5a, 0xad, 0xe1, 0x80, 0x31, 0xbb, 0xe1, 0x80, 0x35, 0x15, 0x50, 0x51, 0x48, 0x01, - 0x2a, 0x35, 0xc8, 0x06, 0x33, 0x4b, 0x6a, 0x38, 0xec, 0xba, 0xb4, 0xdd, 0xf3, 0xd4, 0x38, 0x78, - 0x2e, 0x0c, 0x8d, 0x18, 0x73, 0xd8, 0x0c, 0x6f, 0x67, 0x43, 0xf2, 0x78, 0x85, 0xba, 0x22, 0x62, - 0x2c, 0x54, 0x42, 0x6f, 0x61, 0x51, 0x21, 0x38, 0xbc, 0xa5, 0x9b, 0xf2, 0xc6, 0xd0, 0xe9, 0xf0, - 0x16, 0xf4, 0x00, 0x08, 0xc3, 0x35, 0x89, 0xe7, 0x70, 0x5c, 0x4b, 0x8e, 0x18, 0xb2, 0x1a, 0x1c, - 0x55, 0x67, 0x4c, 0xa3, 0x10, 0x89, 0xf0, 0x70, 0xcc, 0x0f, 0x24, 0x4c, 0x8d, 0x06, 0xd6, 0x99, - 0x51, 0xaa, 0x9b, 0x61, 0x39, 0x1c, 0xf1, 0x96, 0xb4, 0x37, 0x72, 0xb4, 0x04, 0xd5, 0xcd, 0xa9, - 0x46, 0x36, 0xac, 0xea, 0x88, 0x0e, 0x6f, 0xe2, 0x43, 0xe9, 0x2c, 0xce, 0x57, 0xc4, 0x05, 0x20, - 0xd1, 0x05, 0x8f, 0x1b, 0xa7, 0x86, 0x62, 0xf0, 0xdb, 0xca, 0x05, 0x4f, 0x56, 0x05, 0x6b, 0x0d, - 0xd1, 0x00, 0xd6, 0x73, 0x29, 0x13, 0xc7, 0xbe, 0x23, 0xdd, 0xf3, 0x8c, 0xd0, 0xc6, 0xa3, 0xe0, - 0x28, 0xa3, 0xd4, 0x30, 0x28, 0xde, 0xd6, 0xa6, 0xc4, 0x28, 0x73, 0xf5, 0x70, 0x3e, 0x04, 0x0a, - 0xa0, 0x96, 0x47, 0x9a, 0x78, 0x23, 0x77, 0x25, 0x46, 0x59, 0xac, 0xcc, 0xe7, 0x7c, 0x04, 0x24, - 0xfa, 0x1d, 0x5c, 0xd7, 0xb2, 0x27, 0xde, 0xe2, 0x3d, 0xd6, 0xa2, 0x55, 0xc4, 0xc4, 0xa4, 0xe6, - 0x8a, 0xc0, 0xac, 0x86, 0x36, 0xb5, 0xc2, 0xb2, 0x5d, 0x2c, 0x79, 0xda, 0xe8, 0xf0, 0xa4, 0x53, - 0x5c, 0x46, 0x4b, 0x30, 0xd9, 0x64, 0x94, 0x8c, 0x91, 0xa3, 0x0a, 0xe6, 0x25, 0xeb, 0x57, 0x7a, - 0x0e, 0x83, 0x2c, 0xa8, 0xda, 0x54, 0xde, 0x1c, 0xb6, 0x29, 0xef, 0x61, 0x78, 0x65, 0x2c, 0xc9, - 0xac, 0x46, 0x26, 0x27, 0x43, 0x09, 0x1d, 0x47, 0xe2, 0x84, 0x6e, 0x0c, 0x27, 0x82, 0x74, 0xea, - 0xee, 0x2a, 0x23, 0x7b, 0xa9, 0xd4, 0x5d, 0x96, 0xc8, 0x98, 0x30, 0x25, 0xb7, 0x2e, 0x8a, 0xd6, - 0x41, 0xf6, 0xbe, 0x10, 0x19, 0x30, 0x56, 0xef, 0x77, 0x78, 0xe2, 0x8e, 0xfe, 0x64, 0x92, 0xd3, - 0x2e, 0x6b, 0x89, 0x4a, 0x4e, 0xbb, 0x8c, 0x20, 0x5e, 0x84, 0xbe, 0x1d, 0x13, 0x44, 0x5a, 0xb0, - 0xee, 0x68, 0x0e, 0x5c, 0x84, 0x60, 0x9c, 0xfe, 0xe6, 0x78, 0xec, 0xb7, 0xf5, 0xd9, 0xa8, 0x9b, - 0x0a, 0x3a, 0x03, 0xc7, 0x76, 0x18, 0x12, 0x9f, 0x33, 0xe1, 0x0a, 0x8e, 0xcb, 0xd6, 0xc3, 0x91, - 0xfb, 0x4c, 0xdb, 0xe8, 0x7f, 0x94, 0xf2, 0x2f, 0x2c, 0xb2, 0xc3, 0x3d, 0xa3, 0x0c, 0x37, 0xf3, - 0x51, 0x8d, 0x3d, 0x31, 0xdc, 0xbc, 0x48, 0x6b, 0x9e, 0x79, 0x27, 0x2f, 0xec, 0x3e, 0xe1, 0xcb, - 0x41, 0x14, 0xe9, 0x10, 0x3d, 0xf3, 0x4e, 0x1a, 0x7b, 0x8c, 0x1b, 0x8f, 0xe3, 0xa8, 0x80, 0x36, - 0x61, 0x2e, 0x0a, 0xa6, 0xf7, 0x89, 0xdb, 0x26, 0x47, 0x6e, 0xef, 0x92, 0x91, 0xdc, 0x32, 0x56, - 0xc5, 0xe8, 0x36, 0xcc, 0x62, 0xe2, 0x92, 0xaf, 0x13, 0xc5, 0x49, 0xa6, 0xa8, 0x48, 0xad, 0x87, - 0x05, 0x3e, 0xa0, 0x60, 0xe6, 0xdf, 0x66, 0x93, 0x78, 0x9a, 0x99, 0x5f, 0x80, 0x09, 0xaa, 0x10, - 0xf0, 0xb9, 0x8f, 0x0a, 0x74, 0xb0, 0x62, 0x77, 0xca, 0x3e, 0x7b, 0x0c, 0x27, 0x02, 0xba, 0x0a, - 0xb2, 0x1c, 0x58, 0x37, 0x21, 0x0b, 0xba, 0x14, 0xa0, 0xf5, 0xe7, 0x12, 0x94, 0x85, 0x2c, 0x19, - 0xf8, 0x0e, 0xbf, 0xd4, 0x10, 0x45, 0x0a, 0xf8, 0x9c, 0x5c, 0xd2, 0x8e, 0x8d, 0x6d, 0x56, 0x31, - 0xfb, 0x8d, 0xee, 0x45, 0x96, 0x87, 0x5e, 0x27, 0x9a, 0x8d, 0xd9, 0x07, 0xb3, 0x5b, 0xec, 0x1d, - 0x88, 0x90, 0xe2, 0xb8, 0x1e, 0x6d, 0xc0, 0xb4, 0x13, 0x60, 0xdb, 0xed, 0x32, 0x42, 0xc3, 0x26, - 0xa9, 0x8c, 0xd3, 0x22, 0x74, 0x07, 0xa6, 0x9e, 0x7a, 0xbd, 0x0e, 0xf1, 0x03, 0x73, 0x82, 0xdd, - 0xc5, 0xcc, 0x44, 0x60, 0x6f, 0x6c, 0x87, 0x32, 0x69, 0x2c, 0x6a, 0xa9, 0x22, 0x95, 0x51, 0xc5, - 0x49, 0xad, 0x22, 0xaf, 0xb5, 0xbe, 0xd4, 0x5e, 0x04, 0xd0, 0x4f, 0xa9, 0xbb, 0x0d, 0x31, 0xee, - 0xec, 0x37, 0xfa, 0x39, 0x54, 0x85, 0xde, 0x81, 0x13, 0x84, 0xec, 0x33, 0xa7, 0x1f, 0xcc, 0x71, - 0xaf, 0x17, 0x43, 0x48, 0x4a, 0xd6, 0x35, 0x4d, 0x22, 0xd4, 0x3a, 0x83, 0xe9, 0xd6, 0x85, 0xfb, - 0x7e, 0x23, 0x8a, 0xbd, 0xaf, 0xe3, 0x11, 0xa5, 0xbf, 0xd1, 0x7d, 0x98, 0x3a, 0x1a, 0x84, 0xec, - 0x3e, 0x2a, 0xca, 0x82, 0xcf, 0x27, 0x03, 0xca, 0x2b, 0xb0, 0xd0, 0xb0, 0xfe, 0xb3, 0x04, 0x53, - 0xbc, 0x71, 0xf4, 0x39, 0x94, 0xeb, 0x3e, 0xb1, 0x43, 0xb2, 0x13, 0xf2, 0xf7, 0x18, 0xab, 0x5b, - 0xd1, 0xf3, 0x98, 0x2d, 0xf1, 0x3c, 0x26, 0xf5, 0x2a, 0xa3, 0x4c, 0x3d, 0xf5, 0xef, 0xff, 0x67, - 0xbd, 0x84, 0x63, 0x2b, 0xb4, 0x01, 0xe3, 0x34, 0x38, 0x62, 0x2b, 0x6f, 0xfa, 0x41, 0x75, 0x2b, - 0xbc, 0x70, 0xb7, 0x5a, 0x17, 0x2e, 0x95, 0x61, 0x56, 0x43, 0x3f, 0xe5, 0x55, 0x40, 0xfc, 0xd6, - 0x85, 0xcb, 0x3a, 0x57, 0xc6, 0xa2, 0x88, 0x3e, 0x86, 0x0a, 0x1d, 0x73, 0xda, 0xcb, 0xc0, 0x1c, - 0x67, 0x43, 0x87, 0xc4, 0x8d, 0x4d, 0x32, 0x16, 0x38, 0x51, 0x92, 0xf7, 0xff, 0x84, 0xb2, 0xff, - 0xad, 0x2f, 0x74, 0x77, 0x2e, 0xda, 0x79, 0xfb, 0x98, 0x8d, 0xb6, 0x32, 0x6d, 0xb3, 0x49, 0xdb, - 0x0c, 0x20, 0xad, 0x62, 0x2d, 0x6a, 0xb3, 0xce, 0xd6, 0x3f, 0x97, 0xa0, 0x12, 0x0b, 0xa9, 0x3b, - 0x7c, 0xe1, 0x75, 0x48, 0xeb, 0x72, 0x40, 0x78, 0x73, 0x71, 0x99, 0x1e, 0x48, 0xf4, 0x77, 0xa3, - 0xc3, 0x37, 0x29, 0x2f, 0xd1, 0x4f, 0x62, 0x00, 0xcc, 0x28, 0x72, 0x4e, 0x89, 0x80, 0x76, 0xfe, - 0x55, 0x40, 0x3a, 0xdc, 0x3b, 0xb1, 0xdf, 0x54, 0xb6, 0xef, 0x93, 0xe8, 0xda, 0x6d, 0x1c, 0xb3, - 0xdf, 0xb4, 0xe5, 0xa7, 0x4e, 0x88, 0xed, 0xd0, 0xf1, 0x98, 0x03, 0xba, 0x8a, 0xe3, 0xb2, 0xf5, - 0x42, 0x7f, 0x7f, 0x84, 0x1e, 0xc1, 0x4c, 0x2c, 0x64, 0xc3, 0x10, 0xdd, 0x65, 0xc6, 0x57, 0x8e, - 0xb1, 0x81, 0xac, 0x66, 0xf5, 0x60, 0xad, 0x28, 0x05, 0x4b, 0x27, 0xfc, 0x89, 0xef, 0x0d, 0x07, - 0xb1, 0x8b, 0x16, 0xc5, 0x62, 0x07, 0x2d, 0x4e, 0xca, 0x31, 0xf9, 0xa4, 0x7c, 0x08, 0x37, 0x0a, - 0xaf, 0x3d, 0xe4, 0x77, 0x27, 0x13, 0xe2, 0xdd, 0xc9, 0x33, 0xf6, 0xd1, 0x99, 0xa4, 0xee, 0x0f, - 0xe9, 0x9c, 0x75, 0x1f, 0x16, 0xb5, 0xb7, 0x24, 0x74, 0x26, 0xd8, 0x8d, 0x0a, 0x5f, 0x5a, 0xf4, - 0xb7, 0xd5, 0x84, 0xe5, 0x9c, 0x3c, 0x30, 0xaa, 0x01, 0xec, 0xd9, 0xa1, 0x7d, 0x62, 0x07, 0x24, - 0xbe, 0xfe, 0x4d, 0x49, 0x0a, 0x7a, 0xf0, 0x09, 0x98, 0x79, 0x17, 0x2c, 0x05, 0x87, 0xc7, 0x3e, - 0x94, 0xd9, 0xcc, 0x3d, 0x27, 0x97, 0xb4, 0xab, 0xc7, 0x76, 0x78, 0x26, 0xba, 0x4a, 0x7f, 0xd3, - 0x25, 0x79, 0x74, 0x7a, 0x1a, 0x90, 0xe8, 0x35, 0xda, 0x18, 0xe6, 0x25, 0x34, 0x0b, 0x57, 0x9b, - 0xdf, 0xf0, 0x13, 0xe3, 0x6a, 0xf3, 0x1b, 0xeb, 0x11, 0x5f, 0xa2, 0xcc, 0x7b, 0xdf, 0x85, 0xf1, - 0x73, 0xea, 0xd1, 0x4b, 0x92, 0xab, 0x13, 0xf5, 0x3c, 0x9a, 0x63, 0x2a, 0x56, 0x8b, 0x9e, 0xa2, - 0xec, 0xd3, 0xe3, 0x6e, 0x2c, 0xc0, 0x44, 0xc3, 0xed, 0x90, 0x0b, 0x31, 0x59, 0xac, 0x80, 0xee, - 0x27, 0x1d, 0xe5, 0x8e, 0x44, 0xc5, 0xc5, 0xb1, 0x82, 0xf5, 0x46, 0x9b, 0x3b, 0x47, 0x9f, 0x67, - 0x1a, 0xe3, 0x5d, 0x8c, 0x2f, 0xdf, 0xe4, 0x5a, 0xac, 0xaa, 0x5b, 0x47, 0x30, 0x2f, 0x06, 0x35, - 0x46, 0xcf, 0xe9, 0xb0, 0x01, 0x63, 0x4f, 0x1d, 0xf1, 0x88, 0x8f, 0xfe, 0xa4, 0xe3, 0x4b, 0xf5, - 0x79, 0xa4, 0xc5, 0x7e, 0x5b, 0x5f, 0xea, 0x2f, 0xba, 0xd0, 0xbe, 0xa6, 0x21, 0xde, 0x59, 0x33, - 0xb9, 0x56, 0x90, 0xeb, 0x71, 0xd6, 0xc4, 0xc2, 0xda, 0xbc, 0x3f, 0xfa, 0x35, 0x54, 0x63, 0x59, - 0x34, 0x0c, 0xd1, 0x8d, 0x7a, 0xf2, 0x86, 0x32, 0x5d, 0x8d, 0x25, 0x65, 0xbe, 0x6f, 0xb2, 0x57, - 0x62, 0x0f, 0xa0, 0x12, 0x0b, 0xe3, 0xa7, 0x7b, 0x1a, 0x44, 0x9c, 0xa8, 0x59, 0x4d, 0x98, 0x3e, - 0xf6, 0xc9, 0xc0, 0xf6, 0x49, 0x33, 0xec, 0xb3, 0x21, 0x62, 0x11, 0x18, 0x5f, 0x82, 0x2c, 0xfc, - 0x32, 0x60, 0xac, 0xf9, 0xf2, 0x40, 0xc4, 0xac, 0xcd, 0x97, 0x07, 0x74, 0x93, 0x1c, 0xdb, 0xbe, - 0xdd, 0xa7, 0xee, 0x2f, 0xe0, 0xc3, 0x99, 0x92, 0x58, 0x67, 0x60, 0xf0, 0xcb, 0xaa, 0x16, 0xe9, - 0x0f, 0xa2, 0x1b, 0xc5, 0x55, 0x28, 0x8b, 0x6d, 0x24, 0xfc, 0xae, 0x28, 0xd3, 0x09, 0xdc, 0xe9, - 0x39, 0x76, 0x1c, 0x1b, 0xb1, 0x02, 0x0d, 0xf7, 0x8f, 0xcf, 0x2e, 0x03, 0xa7, 0x6d, 0xf7, 0x52, - 0x51, 0xa1, 0x24, 0xb3, 0xfe, 0xad, 0x94, 0xf7, 0x64, 0x81, 0xee, 0x1c, 0x2a, 0x8a, 0x9d, 0x08, - 0x2f, 0xa1, 0x4f, 0xe2, 0x77, 0x88, 0x51, 0xfa, 0x66, 0x4d, 0x7f, 0x9b, 0x17, 0xe9, 0x88, 0x57, - 0x8a, 0xe8, 0x73, 0xb8, 0x1e, 0x7f, 0x4b, 0xa4, 0xe9, 0x78, 0x6e, 0x73, 0x38, 0x18, 0x78, 0x7e, - 0xc8, 0x7d, 0x7f, 0x19, 0x17, 0xa9, 0x3c, 0x1b, 0x2f, 0x8f, 0x19, 0xe3, 0xd6, 0xff, 0x4d, 0xe5, - 0xde, 0x2e, 0xd2, 0x3d, 0xbd, 0xb7, 0xcb, 0x07, 0xe7, 0xea, 0xde, 0x2e, 0x7a, 0x04, 0xd5, 0xd4, - 0xdc, 0x04, 0xfc, 0x08, 0x14, 0xc7, 0x6f, 0xaa, 0x0a, 0x4b, 0x7a, 0xe8, 0x1e, 0x18, 0x07, 0x76, - 0x10, 0xee, 0x9c, 0x9e, 0x92, 0x76, 0x48, 0x3a, 0x2c, 0x14, 0x89, 0x3c, 0x45, 0x46, 0x8e, 0x3e, - 0x83, 0x59, 0x7a, 0xd4, 0x1f, 0x90, 0x77, 0xa4, 0x97, 0x3e, 0xe4, 0x17, 0xc4, 0x9d, 0x71, 0xba, - 0x12, 0x2b, 0xba, 0x68, 0x0f, 0x6e, 0xc8, 0x0a, 0xa4, 0x47, 0xec, 0x80, 0x24, 0xe3, 0x12, 0x45, - 0xe4, 0xc5, 0x4a, 0x68, 0x1f, 0xe6, 0xa8, 0xc2, 0x1e, 0x39, 0x75, 0x5c, 0xd2, 0x79, 0x6d, 0xc7, - 0xd1, 0x9f, 0x32, 0x35, 0xb2, 0x12, 0x56, 0x8d, 0xd0, 0x2f, 0x60, 0x59, 0x11, 0x3d, 0xbe, 0xe0, - 0xfd, 0x98, 0x62, 0xfd, 0xc8, 0xab, 0xa6, 0x3d, 0x68, 0x5e, 0x06, 0x21, 0xe9, 0xbf, 0xb6, 0x7d, - 0x87, 0xce, 0x5f, 0x60, 0x96, 0x75, 0x3d, 0x90, 0x95, 0xb0, 0x6a, 0x44, 0x7b, 0xa0, 0x88, 0xe2, - 0x1e, 0x54, 0xa2, 0x1e, 0xe4, 0x54, 0xa3, 0xcf, 0x60, 0x45, 0xe9, 0x1c, 0x26, 0x83, 0x9e, 0x7d, - 0xc9, 0xae, 0xf1, 0x81, 0xd9, 0xe6, 0x2b, 0x50, 0x6b, 0x05, 0x38, 0x65, 0x3d, 0x1d, 0x59, 0xe7, - 0x2a, 0xa0, 0xa7, 0xb0, 0xae, 0x54, 0x36, 0x5d, 0x7b, 0x10, 0x9c, 0x79, 0x61, 0xcb, 0xf3, 0x0e, - 0x6c, 0xbf, 0x4b, 0x58, 0x1a, 0xa9, 0x8c, 0x47, 0xa9, 0x51, 0x24, 0xa5, 0x93, 0x19, 0xa4, 0x99, - 0x08, 0x69, 0x84, 0x1a, 0xdd, 0x6f, 0x7c, 0x4d, 0x77, 0xe8, 0xa2, 0x3e, 0xf0, 0xdc, 0x2e, 0x75, - 0x17, 0xf5, 0x33, 0xd2, 0x3e, 0x27, 0x1d, 0x96, 0x42, 0x2a, 0xe3, 0x22, 0x15, 0x1a, 0xb4, 0xed, - 0x7b, 0x43, 0x37, 0x5a, 0xfe, 0x73, 0x51, 0x1e, 0x37, 0x16, 0xa0, 0x4f, 0x01, 0xe2, 0xcd, 0x1a, - 0x98, 0x06, 0x9b, 0xec, 0x65, 0x79, 0xb2, 0xe3, 0x7a, 0x9c, 0x52, 0x45, 0x8f, 0x60, 0x29, 0x2e, - 0x51, 0x37, 0x4a, 0xe2, 0x19, 0x9e, 0x67, 0x7d, 0xca, 0xa9, 0xb5, 0xfe, 0x6b, 0x42, 0xff, 0x0a, - 0x2a, 0xd7, 0x4f, 0x45, 0xde, 0xe0, 0x6a, 0xec, 0x0d, 0x36, 0x60, 0xba, 0x49, 0xc2, 0xd7, 0xb6, - 0x1f, 0x39, 0x83, 0x31, 0x46, 0xe5, 0xd3, 0xa2, 0x8c, 0xbf, 0x18, 0xff, 0x11, 0xfe, 0x62, 0xe2, - 0xbd, 0xfd, 0xc5, 0xe4, 0xf7, 0xf0, 0x17, 0x9a, 0x9d, 0x3e, 0xf5, 0x13, 0xef, 0xf4, 0xf2, 0xf7, - 0xde, 0xe9, 0x95, 0x9f, 0x78, 0xa7, 0xc3, 0x8f, 0xd8, 0xe9, 0xd3, 0x3f, 0x6a, 0xa7, 0x57, 0x47, - 0xed, 0x74, 0x69, 0x4f, 0xcc, 0x14, 0xef, 0x89, 0xd9, 0xf7, 0xde, 0x13, 0xd6, 0x1f, 0x4a, 0x71, - 0x62, 0x4b, 0xee, 0xb9, 0x36, 0x9e, 0xd8, 0x80, 0x89, 0xd7, 0x76, 0x6f, 0x48, 0x78, 0x18, 0x09, - 0x5b, 0xec, 0xef, 0x44, 0x1e, 0x5f, 0x0c, 0x7c, 0x1c, 0x55, 0xb0, 0x88, 0xe3, 0xab, 0x1e, 0x3f, - 0xf0, 0xe9, 0x4f, 0x16, 0xe2, 0x05, 0xbb, 0x8e, 0xcb, 0x0f, 0xda, 0xa8, 0x40, 0x17, 0x2e, 0x5f, - 0xc8, 0x2c, 0xf8, 0x78, 0x4e, 0xe3, 0xfc, 0x88, 0x71, 0x66, 0xe4, 0xa8, 0x06, 0xe3, 0x8c, 0xbe, - 0x4d, 0xa6, 0x1b, 0xa5, 0x12, 0xcc, 0xe4, 0xd6, 0x4b, 0x98, 0x91, 0x16, 0xab, 0xb6, 0xeb, 0x31, - 0x8f, 0x89, 0x62, 0xff, 0xa8, 0xc0, 0xfe, 0x98, 0xe4, 0xc2, 0x6d, 0xec, 0x45, 0x9b, 0xb2, 0x8a, - 0x79, 0xc9, 0xfa, 0x9b, 0x9c, 0x74, 0x5f, 0x01, 0x1d, 0xf8, 0x35, 0xac, 0x8f, 0x78, 0xb2, 0x98, - 0x66, 0x20, 0x25, 0x99, 0x81, 0x58, 0xb0, 0x31, 0x2a, 0xc7, 0x67, 0x6d, 0xb2, 0x97, 0xa3, 0x9a, - 0x37, 0x87, 0xd4, 0xdf, 0xd4, 0x5f, 0x88, 0xe8, 0xa3, 0xfe, 0x82, 0xff, 0x19, 0x81, 0x2e, 0x1b, - 0x97, 0xf3, 0x67, 0x04, 0x1f, 0x69, 0x5f, 0x27, 0xe6, 0xf9, 0x37, 0xeb, 0x58, 0x9f, 0xbb, 0xcb, - 0x1f, 0x1c, 0x1a, 0x76, 0xee, 0x0c, 0xc3, 0xb3, 0x66, 0xe8, 0x3b, 0x6e, 0x74, 0x87, 0x5a, 0xc5, - 0x29, 0x89, 0xb5, 0xad, 0x79, 0xd0, 0x48, 0xe3, 0x4e, 0x21, 0x12, 0x7f, 0x6e, 0x21, 0xca, 0xd6, - 0xc7, 0xba, 0x6c, 0x5f, 0xa1, 0xc5, 0x2f, 0x35, 0xaf, 0x1c, 0xd1, 0x2d, 0x98, 0x11, 0xa2, 0xdd, - 0xcb, 0x90, 0x04, 0x7c, 0x58, 0x64, 0xa1, 0xf5, 0x2b, 0x5d, 0x26, 0xf0, 0x3d, 0x6d, 0xcf, 0x72, - 0x9f, 0x43, 0xa2, 0x6d, 0xbe, 0xae, 0x4b, 0x2c, 0x98, 0xbd, 0x9e, 0x93, 0x93, 0x4b, 0x16, 0x7a, - 0x7c, 0x99, 0xd1, 0x74, 0xbe, 0x21, 0x9c, 0x54, 0x26, 0x02, 0xeb, 0x22, 0x3f, 0xbd, 0x28, 0x5b, - 0x96, 0x14, 0x4b, 0xfa, 0x25, 0xac, 0x50, 0xb7, 0x07, 0x76, 0xdb, 0x09, 0x2f, 0x39, 0xb6, 0x2c, - 0xa4, 0xb3, 0x7b, 0x48, 0x82, 0xc0, 0xee, 0xc6, 0xb7, 0xbc, 0xbc, 0x68, 0x1d, 0x15, 0x3f, 0xc9, - 0xfc, 0xde, 0x1f, 0x6a, 0xfd, 0xfd, 0x88, 0x34, 0xe6, 0x5f, 0xf9, 0x7b, 0x3e, 0xd1, 0xbf, 0xf9, - 0x2c, 0x6e, 0xd5, 0xfa, 0xdb, 0x9c, 0x4c, 0x68, 0xb6, 0x3b, 0x25, 0x4d, 0x77, 0xac, 0xff, 0x2f, - 0x8d, 0x78, 0xed, 0x37, 0xe2, 0x7a, 0x9e, 0x75, 0x2e, 0xb4, 0x7b, 0x5e, 0x37, 0xbe, 0xe0, 0x48, - 0x04, 0xb4, 0x96, 0xba, 0x41, 0x96, 0x05, 0x14, 0xf7, 0x60, 0xb1, 0x80, 0x7a, 0x85, 0xe8, 0xd5, - 0xc9, 0x78, 0xc4, 0xe2, 0x22, 0xde, 0x57, 0x03, 0x10, 0xc1, 0x1d, 0xbf, 0x0f, 0x1c, 0xc3, 0x29, - 0x09, 0x7a, 0x90, 0x0c, 0xd3, 0x81, 0xd7, 0x66, 0x9c, 0xe9, 0xa9, 0x1d, 0x9c, 0x31, 0x3f, 0x5d, - 0xc1, 0xda, 0x3a, 0xba, 0x43, 0xa3, 0x37, 0x4b, 0x8d, 0x3d, 0x16, 0xd9, 0x57, 0x70, 0x5c, 0xb6, - 0x9e, 0x8e, 0x4a, 0xb4, 0x45, 0xe9, 0x80, 0xbe, 0xf7, 0x8e, 0x74, 0x1e, 0xbb, 0xa1, 0xef, 0xc4, - 0x7b, 0x4e, 0x91, 0x5a, 0x5b, 0xba, 0xd7, 0xb5, 0x74, 0xc2, 0xb9, 0x84, 0x3b, 0x07, 0x51, 0xb4, - 0xb6, 0xb5, 0xa9, 0xe6, 0x02, 0x83, 0x03, 0xdd, 0x6b, 0x5b, 0xea, 0x2f, 0xf9, 0xf3, 0x42, 0xfe, - 0x07, 0x5f, 0xfc, 0x25, 0xa1, 0x20, 0xdd, 0x84, 0x5d, 0x8f, 0x47, 0x71, 0x61, 0x4a, 0x62, 0xdd, - 0xd5, 0x66, 0xa5, 0xb5, 0xe9, 0x82, 0x7b, 0x79, 0x6f, 0x73, 0xb3, 0x79, 0x0b, 0xeb, 0xa3, 0xdc, - 0xf4, 0xb4, 0x16, 0xba, 0x5f, 0xf0, 0x32, 0x17, 0x6d, 0xc2, 0x1c, 0xff, 0x3b, 0xc1, 0x38, 0xe7, - 0x11, 0x1d, 0x61, 0xaa, 0x98, 0xce, 0xd1, 0x1b, 0xdf, 0x09, 0x13, 0x08, 0xbe, 0x18, 0x15, 0xa9, - 0xe5, 0x16, 0x25, 0xb7, 0xff, 0x0a, 0xed, 0xbd, 0xd6, 0x67, 0xba, 0xd1, 0xdf, 0x41, 0x35, 0x2d, - 0x7f, 0x8f, 0x3f, 0x9c, 0x94, 0xf4, 0xad, 0x7f, 0x28, 0x15, 0xbd, 0x99, 0x1d, 0xb1, 0x69, 0x2d, - 0xa8, 0xd2, 0x33, 0x95, 0xb0, 0x9b, 0x8a, 0x78, 0xdf, 0x4a, 0x32, 0x1a, 0x2a, 0xf1, 0xdb, 0xda, - 0xc7, 0x17, 0xed, 0xde, 0x30, 0x70, 0xde, 0x11, 0x7e, 0x8b, 0x9b, 0x91, 0x5b, 0x9f, 0x16, 0xa6, - 0x8d, 0x0b, 0xa2, 0x97, 0x7f, 0x4d, 0xe2, 0x40, 0x39, 0x06, 0xfd, 0x81, 0x71, 0xe0, 0x26, 0xcc, - 0xbd, 0x20, 0x17, 0x61, 0xcb, 0xb7, 0xdd, 0xc0, 0x8e, 0xee, 0x6c, 0xa2, 0xf4, 0x84, 0x2a, 0x46, - 0x5b, 0x50, 0xc5, 0x43, 0x97, 0x8e, 0x76, 0x04, 0x39, 0x9e, 0x81, 0x94, 0xea, 0xef, 0xfd, 0xcb, - 0x64, 0xea, 0x1d, 0x2f, 0xaa, 0xf0, 0xbf, 0x1e, 0x36, 0xae, 0xa0, 0x6b, 0x30, 0xa7, 0x3c, 0xad, - 0x35, 0x4a, 0xc8, 0x80, 0x6a, 0x3a, 0x21, 0x6d, 0x5c, 0x45, 0x55, 0x28, 0x8b, 0xdc, 0xb0, 0x31, - 0x86, 0x66, 0xa0, 0x12, 0xe7, 0xe9, 0x8c, 0x71, 0x34, 0x07, 0xd3, 0xa9, 0xe4, 0x94, 0x31, 0x81, - 0x66, 0x01, 0x92, 0xa4, 0x87, 0x31, 0x49, 0xf1, 0xd2, 0xb7, 0xfd, 0xc6, 0x14, 0xd5, 0x48, 0x5e, - 0x70, 0x1a, 0x65, 0x8a, 0x18, 0x3f, 0xcc, 0x34, 0x2a, 0x68, 0x49, 0xf7, 0x34, 0xd3, 0x00, 0x2a, - 0xcf, 0x3e, 0x91, 0x34, 0xa6, 0x11, 0x52, 0x1f, 0x49, 0x1a, 0x55, 0x34, 0x1d, 0xbf, 0x78, 0x34, - 0x66, 0xd0, 0x4a, 0xce, 0x63, 0x46, 0x63, 0x16, 0xcd, 0x2b, 0x6f, 0x11, 0x8d, 0x39, 0xb4, 0x90, - 0x7d, 0x5a, 0x68, 0x18, 0xe9, 0xaf, 0xa0, 0xbc, 0xdb, 0x98, 0xe7, 0x92, 0xf8, 0x72, 0xd1, 0x40, - 0x74, 0x38, 0x95, 0x67, 0x76, 0xc6, 0x35, 0x2a, 0x54, 0xae, 0xc0, 0x8c, 0x05, 0xda, 0xac, 0x14, - 0x2c, 0x1b, 0x8b, 0x68, 0x2d, 0xff, 0x69, 0x9b, 0xb1, 0x44, 0x87, 0x28, 0x4e, 0x91, 0x1b, 0xcb, - 0xbc, 0xa5, 0x74, 0xb8, 0x6a, 0x98, 0xb4, 0x43, 0xe9, 0x18, 0xd3, 0x58, 0x61, 0x53, 0x71, 0x74, - 0xb8, 0xf3, 0xf6, 0x18, 0x1f, 0xd5, 0x9b, 0xc6, 0x2a, 0x2f, 0x3f, 0x3e, 0x3c, 0x68, 0x1c, 0x36, - 0x5a, 0xc6, 0x75, 0xfa, 0xa9, 0x6a, 0xd0, 0x60, 0xac, 0xd1, 0xe1, 0xd2, 0x86, 0x12, 0xc6, 0x0d, - 0xd6, 0xef, 0xf4, 0x81, 0x6d, 0xd4, 0xd8, 0xfc, 0x1f, 0xc5, 0x07, 0x81, 0xb1, 0x4e, 0x05, 0x29, - 0xd7, 0x6c, 0x6c, 0xd0, 0xce, 0x2a, 0x4e, 0xd5, 0xb8, 0x49, 0x27, 0x33, 0xeb, 0xcb, 0x0c, 0x8b, - 0x7e, 0x44, 0xda, 0x57, 0x18, 0x1f, 0xa0, 0xeb, 0xcc, 0x27, 0xeb, 0xd2, 0xf6, 0xc6, 0x2d, 0xb4, - 0x08, 0xf3, 0x99, 0x2c, 0xb6, 0xf1, 0x21, 0x5a, 0x85, 0x25, 0xfd, 0xb9, 0x68, 0xdc, 0x46, 0xcb, - 0x70, 0x4d, 0xb3, 0xe1, 0x8d, 0x3b, 0xf7, 0xfe, 0xb1, 0x24, 0x51, 0x98, 0xe4, 0x5e, 0x94, 0x8e, - 0x88, 0x52, 0x11, 0x11, 0x5c, 0xe3, 0x0a, 0xba, 0x0f, 0x77, 0x94, 0xaa, 0xe6, 0xb9, 0x33, 0xd0, - 0x5d, 0x01, 0x1a, 0x25, 0xf4, 0x11, 0xdc, 0x55, 0x71, 0x5c, 0x3a, 0x28, 0x5a, 0xf5, 0xab, 0xf7, - 0x7e, 0x06, 0x0b, 0xba, 0x88, 0x0f, 0x01, 0x3d, 0x34, 0xfb, 0x1e, 0xdb, 0xc2, 0x65, 0x18, 0xdf, - 0x73, 0x82, 0x73, 0xa3, 0xb4, 0x7b, 0xf0, 0xc7, 0x6f, 0x6b, 0xa5, 0x3f, 0x7d, 0x5b, 0x2b, 0xfd, - 0xef, 0xb7, 0xb5, 0x2b, 0xbf, 0xff, 0xae, 0x76, 0xe5, 0x9f, 0xbe, 0xab, 0x95, 0xfe, 0xf4, 0x5d, - 0xed, 0xca, 0x9f, 0xbf, 0xab, 0x5d, 0xf9, 0x62, 0x2b, 0xf5, 0x7f, 0x12, 0xf4, 0xed, 0xd0, 0x77, - 0x2e, 0x3c, 0xdf, 0xe9, 0x3a, 0xae, 0x28, 0xb8, 0x64, 0x7b, 0x70, 0xde, 0xdd, 0x1e, 0x9c, 0x6c, - 0xb3, 0x50, 0xf3, 0x64, 0x92, 0xa5, 0x5b, 0x7f, 0xfe, 0x97, 0x00, 0x00, 0x00, 0xff, 0xff, 0x2a, - 0x1e, 0x46, 0xf1, 0x33, 0x41, 0x00, 0x00, + // 4557 bytes of a gzipped FileDescriptorProto + 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x5c, 0xcb, 0x73, 0xdb, 0x48, + 0x7a, 0x37, 0xf5, 0x24, 0x3f, 0x51, 0x12, 0xd4, 0xd6, 0x03, 0x92, 0x25, 0x4a, 0xc6, 0x78, 0x6c, + 0xd9, 0xde, 0x91, 0x26, 0xda, 0x19, 0xcf, 0x3e, 0x26, 0xa9, 0x91, 0x48, 0x4b, 0xa6, 0x4d, 0x59, + 0x72, 0x93, 0x7e, 0xec, 0x6c, 0xd5, 0x54, 0x41, 0x64, 0x8b, 0xc2, 0x8a, 0x04, 0x38, 0x00, 0xe8, + 0x91, 0xa6, 0x2a, 0xff, 0x40, 0xaa, 0x92, 0xda, 0x3f, 0x21, 0x39, 0xe5, 0x96, 0xe3, 0xde, 0xf7, + 0x92, 0xda, 0xe3, 0x1e, 0x52, 0x95, 0xcd, 0x25, 0x49, 0xcd, 0xfc, 0x0f, 0xb9, 0x66, 0xab, 0x1b, + 0xdd, 0x00, 0x1a, 0x68, 0x80, 0x9e, 0xd7, 0x45, 0x85, 0xfe, 0xfa, 0xfb, 0x7e, 0xfd, 0xfe, 0xfa, + 0x7b, 0x34, 0x05, 0x33, 0x5f, 0x0e, 0x89, 0x7b, 0xbd, 0x33, 0x70, 0x1d, 0xdf, 0x41, 0x93, 0xac, + 0xb0, 0x56, 0xf6, 0x7c, 0xd3, 0x1f, 0x7a, 0x01, 0x71, 0x0d, 0x7a, 0x4e, 0xfb, 0x92, 0x7f, 0x97, + 0xfc, 0x2b, 0x9b, 0x7f, 0xce, 0xfb, 0x56, 0x9f, 0x78, 0xbe, 0xd9, 0x1f, 0x08, 0x02, 0x95, 0xf2, + 0x2c, 0xfb, 0xdc, 0x11, 0x82, 0x83, 0x9e, 0x29, 0xb8, 0x3f, 0xe8, 0x5a, 0xfe, 0xc5, 0xf0, 0x6c, + 0xa7, 0xed, 0xf4, 0x77, 0xbb, 0x4e, 0xd7, 0xd9, 0x65, 0xe4, 0xb3, 0xe1, 0x39, 0x2b, 0xb1, 0x02, + 0xfb, 0xe2, 0xec, 0x9b, 0x5d, 0xc7, 0xe9, 0xf6, 0x48, 0xc4, 0x95, 0x68, 0xcc, 0xb8, 0x03, 0xe5, + 0x17, 0xb4, 0xaf, 0x98, 0x7c, 0x39, 0x24, 0x9e, 0x8f, 0x16, 0x61, 0x92, 0x95, 0xf5, 0xc2, 0x56, + 0x61, 0xbb, 0x84, 0x83, 0x82, 0xf1, 0x1c, 0x96, 0x9b, 0x17, 0xce, 0x57, 0xa7, 0xae, 0xd3, 0x26, + 0x9e, 0xd7, 0xb0, 0x3c, 0x5f, 0xf0, 0x2f, 0xc3, 0x54, 0x8b, 0xd8, 0xa6, 0xed, 0x73, 0x01, 0x5e, + 0x42, 0xeb, 0x50, 0x6a, 0x5e, 0x7b, 0xbc, 0x6a, 0x6c, 0xab, 0xb0, 0x5d, 0xc4, 0x11, 0xc1, 0x78, + 0x0d, 0x0b, 0xcd, 0x6b, 0xbb, 0x5d, 0x75, 0xfa, 0x7d, 0x2b, 0x84, 0x3a, 0x80, 0xb9, 0x86, 0xe9, + 0x13, 0xcf, 0x0f, 0xc8, 0xad, 0x26, 0x83, 0x9c, 0xd9, 0x5b, 0xdc, 0x89, 0x3a, 0xdd, 0x12, 0x5f, + 0x07, 0x13, 0x7f, 0xfa, 0xef, 0xcd, 0x1b, 0x38, 0x21, 0x61, 0x7c, 0x0e, 0x28, 0x0e, 0xec, 0x0d, + 0x1c, 0xdb, 0x23, 0xa8, 0x06, 0xf3, 0xd5, 0xa1, 0xeb, 0x12, 0xfb, 0xbb, 0x40, 0x27, 0x45, 0x0c, + 0x04, 0xda, 0x11, 0xf1, 0xa5, 0x3e, 0x1b, 0xbf, 0x81, 0x85, 0x18, 0xed, 0x47, 0x6d, 0x6e, 0x17, + 0x96, 0xaa, 0x8e, 0x4b, 0x6a, 0xc3, 0xfe, 0xa0, 0xea, 0xd8, 0xe7, 0x56, 0x37, 0x36, 0xe5, 0xfb, + 0x6d, 0xdf, 0x72, 0x6c, 0x31, 0xe5, 0x41, 0xc9, 0xd0, 0x61, 0x39, 0x29, 0x10, 0x74, 0xc8, 0xb8, + 0x05, 0xab, 0x47, 0xc4, 0x3f, 0xa5, 0x0b, 0xde, 0x76, 0x7a, 0xaf, 0x88, 0xeb, 0x59, 0x8e, 0x2d, + 0x86, 0xf0, 0xef, 0x05, 0x58, 0x53, 0xd5, 0xf2, 0xc1, 0xe8, 0x30, 0xcd, 0x49, 0xac, 0xb9, 0x71, + 0x2c, 0x8a, 0xe8, 0x09, 0x6c, 0xd6, 0x6a, 0x8d, 0x57, 0x96, 0x67, 0x9d, 0x59, 0x3d, 0xcb, 0xbf, + 0xa6, 0xdd, 0x78, 0x6b, 0xd2, 0xae, 0x9c, 0xba, 0x64, 0x60, 0xba, 0xa4, 0xc3, 0x17, 0x7e, 0x14, + 0x1b, 0xaa, 0xc1, 0x46, 0x06, 0xcb, 0x21, 0xb1, 0xdb, 0xa4, 0xa3, 0x8f, 0x33, 0x9c, 0x7c, 0x26, + 0xe3, 0x1f, 0xc7, 0x60, 0xb5, 0x99, 0x35, 0xcc, 0x9c, 0x71, 0x1c, 0x42, 0x25, 0x03, 0xb8, 0x65, + 0xba, 0x5d, 0xe2, 0x7b, 0xfa, 0xd8, 0xd6, 0xf8, 0x76, 0x09, 0x8f, 0xe0, 0x4a, 0x8d, 0x22, 0xa0, + 0x1f, 0x11, 0x9b, 0xb8, 0x8c, 0x8f, 0x8d, 0x62, 0x02, 0xe7, 0x33, 0xa5, 0x66, 0x35, 0x60, 0x60, + 0xc7, 0x70, 0xbf, 0xd3, 0x71, 0x89, 0xe7, 0xe9, 0x13, 0x6c, 0xd9, 0x47, 0xb1, 0x19, 0x8f, 0x60, + 0xad, 0xf9, 0x3d, 0xd6, 0xd5, 0xf8, 0x19, 0xcc, 0xe1, 0xa1, 0xdd, 0x32, 0xbd, 0x4b, 0x31, 0x77, + 0x6b, 0x50, 0xa4, 0xc5, 0xaa, 0xd3, 0x21, 0x8c, 0x79, 0x12, 0x87, 0x65, 0xe3, 0x3e, 0xcc, 0x87, + 0xdc, 0x1c, 0x7a, 0x19, 0xa6, 0x30, 0xf1, 0x86, 0xbd, 0x50, 0x27, 0x04, 0x25, 0xba, 0x41, 0xe9, + 0x46, 0xb3, 0x06, 0xa4, 0x67, 0xd9, 0xa4, 0x6e, 0x9f, 0x3b, 0x62, 0x0f, 0xee, 0xc2, 0x4a, 0xaa, + 0x86, 0x83, 0x2d, 0xc2, 0x64, 0xd5, 0x19, 0x72, 0xfd, 0x32, 0x8e, 0x83, 0x82, 0xf1, 0x87, 0x55, + 0x98, 0x16, 0xbd, 0x5b, 0x87, 0x12, 0xff, 0xac, 0xd7, 0x18, 0xd7, 0x04, 0x8e, 0x08, 0x68, 0x07, + 0x4a, 0xd5, 0x7e, 0xe7, 0x98, 0xf8, 0x17, 0x4e, 0xb0, 0x1f, 0xe7, 0xf6, 0xb4, 0x9d, 0x40, 0x57, + 0x87, 0x74, 0x1c, 0xb1, 0xa0, 0x4f, 0x64, 0x85, 0xc8, 0x16, 0x6d, 0x66, 0xef, 0x26, 0x17, 0x89, + 0x57, 0x61, 0x59, 0x73, 0xbe, 0xcc, 0xd2, 0x91, 0x6c, 0xbd, 0x66, 0xf6, 0x36, 0x38, 0x84, 0x9a, + 0x09, 0x67, 0x29, 0xd8, 0x06, 0xdc, 0xdc, 0xef, 0xf9, 0xc4, 0xdd, 0x6f, 0xb7, 0xe9, 0xc8, 0x05, + 0xe6, 0x24, 0xc3, 0x5c, 0xe3, 0x98, 0x0a, 0x0e, 0xac, 0x12, 0x43, 0x9f, 0xc1, 0xfc, 0x33, 0xab, + 0xd7, 0xab, 0x3a, 0xb6, 0x38, 0x18, 0xfa, 0x14, 0x43, 0x5a, 0xe6, 0x48, 0x89, 0x5a, 0x9c, 0x64, + 0x47, 0x55, 0xd0, 0x5a, 0xae, 0xd9, 0x26, 0xcd, 0x81, 0x19, 0x42, 0x4c, 0x33, 0x88, 0x15, 0x0e, + 0x91, 0xac, 0xc6, 0x29, 0x01, 0x54, 0x07, 0x74, 0x44, 0xfc, 0x86, 0xd3, 0xbe, 0x8c, 0xed, 0x02, + 0xbd, 0xc8, 0x60, 0x56, 0x39, 0x4c, 0x9a, 0x01, 0x2b, 0x84, 0xd0, 0x21, 0xd3, 0xc0, 0xad, 0x2b, + 0x3b, 0x8e, 0x54, 0x62, 0x48, 0x7a, 0x84, 0x24, 0xd7, 0xe3, 0xb4, 0x08, 0x9d, 0x67, 0xaa, 0xc9, + 0xcd, 0xf6, 0x45, 0x7c, 0x67, 0xea, 0x20, 0xcd, 0xb3, 0x82, 0x03, 0xab, 0xc4, 0xd0, 0x2f, 0x00, + 0x9a, 0xd7, 0x6d, 0x3b, 0x50, 0xe6, 0xfa, 0x8c, 0xd4, 0x9d, 0xd4, 0xcd, 0x87, 0x63, 0xbc, 0xe8, + 0x63, 0x28, 0x85, 0x37, 0x8a, 0x5e, 0x96, 0x26, 0x36, 0x79, 0xfb, 0xe0, 0x88, 0x13, 0x9d, 0xb2, + 0x19, 0x4d, 0x1c, 0x76, 0x7d, 0x96, 0xc9, 0x6f, 0x45, 0xf2, 0x6a, 0xe5, 0x88, 0x15, 0xb2, 0x14, + 0x31, 0xad, 0x3e, 0xf4, 0x39, 0x09, 0xb1, 0x99, 0x8d, 0x98, 0xae, 0x42, 0x35, 0x98, 0x93, 0x2f, + 0x28, 0x7d, 0x9e, 0xa1, 0xad, 0x8b, 0xf3, 0xa8, 0xba, 0xee, 0x70, 0x42, 0x06, 0xed, 0xc2, 0x34, + 0x57, 0x38, 0xba, 0xc6, 0xc4, 0x97, 0xb8, 0xb8, 0xac, 0xb4, 0xb0, 0xe0, 0x42, 0xbf, 0x81, 0x25, + 0x4c, 0xfa, 0xce, 0x5b, 0x42, 0xff, 0xfa, 0x84, 0x6e, 0xa0, 0x96, 0x79, 0xd6, 0x23, 0xfa, 0x02, + 0x13, 0x7f, 0x4f, 0x88, 0xab, 0x78, 0x04, 0x98, 0x1a, 0x01, 0xed, 0xc3, 0x2c, 0xdd, 0x92, 0xcc, + 0x06, 0x39, 0xb0, 0xec, 0x8e, 0x8e, 0x18, 0xe4, 0xad, 0xd8, 0x16, 0x0e, 0xeb, 0x04, 0x94, 0x2c, + 0x81, 0x9e, 0x82, 0xf6, 0xd2, 0xf6, 0x86, 0x67, 0x5e, 0xdb, 0xb5, 0xce, 0x48, 0xd0, 0xb1, 0x9b, + 0x0c, 0xa5, 0xc2, 0x51, 0x92, 0xd5, 0xe1, 0xb1, 0x4a, 0x56, 0xc4, 0xf7, 0x70, 0xcd, 0xf4, 0x4d, + 0xb1, 0x87, 0x17, 0x95, 0x7b, 0x38, 0xc6, 0x81, 0x55, 0x62, 0x1c, 0xad, 0x49, 0x8d, 0xd1, 0xf8, + 0x89, 0x58, 0x4a, 0xa2, 0x25, 0x39, 0xb0, 0x4a, 0x8c, 0xaa, 0x47, 0xb5, 0xf2, 0xd7, 0x97, 0x25, + 0xf5, 0xa8, 0x66, 0xc2, 0x19, 0xc2, 0x14, 0xf6, 0xd8, 0xea, 0xba, 0xa6, 0x4f, 0xa8, 0x92, 0x3a, + 0x74, 0x9d, 0xbe, 0x80, 0x5d, 0x91, 0x60, 0xd5, 0x4c, 0x38, 0x43, 0x18, 0x9d, 0xc0, 0x62, 0xac, + 0xa6, 0x15, 0xf6, 0x55, 0x97, 0xd6, 0x57, 0xc5, 0x82, 0x95, 0x82, 0xe8, 0x0c, 0x74, 0x4c, 0x7a, + 0x8e, 0xd9, 0xd9, 0x1f, 0xfa, 0x4e, 0xdd, 0x6e, 0xbb, 0xa4, 0x4f, 0x8d, 0x3d, 0x3a, 0xe7, 0xfa, + 0x2a, 0x03, 0xbd, 0x1b, 0xee, 0x43, 0x35, 0x9b, 0xc0, 0xcf, 0xc4, 0xa1, 0xaa, 0xb9, 0xea, 0xf7, + 0x30, 0x31, 0x3b, 0xc4, 0x15, 0x1d, 0x5e, 0x93, 0x34, 0x48, 0xb2, 0x1a, 0xa7, 0x04, 0xd0, 0x31, + 0xcc, 0x1f, 0x11, 0x1f, 0x93, 0x41, 0xcf, 0x6a, 0x9b, 0xc1, 0xcd, 0x7b, 0x2b, 0xb9, 0x40, 0xf1, + 0x5a, 0x2e, 0x27, 0xac, 0xd8, 0x44, 0x2d, 0xdd, 0x44, 0x98, 0x78, 0xc4, 0x6f, 0x12, 0x2f, 0xa6, + 0x1e, 0xf4, 0x75, 0x69, 0x13, 0x29, 0x38, 0xb0, 0x4a, 0x0c, 0x35, 0x60, 0xe1, 0xc8, 0x39, 0x36, + 0xaf, 0xe8, 0x3d, 0xe9, 0x09, 0xac, 0x0d, 0x59, 0xd9, 0x27, 0xeb, 0x79, 0xcf, 0xd2, 0x82, 0x1c, + 0x8d, 0xf4, 0x1b, 0x56, 0xa4, 0x53, 0xf5, 0x4a, 0x12, 0x4d, 0xae, 0x8f, 0xa1, 0xc9, 0x15, 0xe8, + 0x0b, 0x58, 0x39, 0xb4, 0x7a, 0xa4, 0x49, 0xdc, 0xb7, 0x56, 0x9b, 0xc4, 0x97, 0x4c, 0xdf, 0x94, + 0xce, 0x73, 0x06, 0x17, 0x47, 0xce, 0x02, 0x41, 0x7d, 0x58, 0x4f, 0x56, 0x3d, 0x7e, 0x6b, 0xb5, + 0xc3, 0x8e, 0x6f, 0x49, 0xda, 0x2c, 0x8f, 0x95, 0xb7, 0x94, 0x0b, 0x87, 0x5e, 0xc2, 0xe2, 0x31, + 0xf1, 0xcd, 0x8e, 0xe9, 0x9b, 0xd2, 0x58, 0x6e, 0xcb, 0x27, 0x40, 0xc1, 0xc2, 0xe1, 0x95, 0xe2, + 0xe8, 0x04, 0xd0, 0x91, 0x73, 0x54, 0x3d, 0x25, 0x6e, 0x9b, 0x44, 0xd6, 0x8c, 0x21, 0xdf, 0xfc, + 0x29, 0x06, 0x0e, 0xa9, 0x10, 0xa5, 0xa6, 0xc4, 0xa1, 0x39, 0xec, 0xf9, 0x75, 0xfb, 0x77, 0x24, + 0x9a, 0x8c, 0xf7, 0x24, 0xc0, 0x34, 0x03, 0x56, 0x08, 0xa1, 0xdf, 0xc2, 0x72, 0xd5, 0xef, 0x1d, + 0x3b, 0x4c, 0x99, 0x32, 0x05, 0x26, 0xe0, 0xee, 0x48, 0x27, 0x40, 0xcd, 0xc4, 0xfb, 0x98, 0x01, + 0x81, 0xbe, 0x80, 0xd5, 0xd7, 0x8e, 0x7b, 0xe9, 0x0d, 0xcc, 0x36, 0x69, 0x5d, 0xb8, 0xc4, 0xbb, + 0x70, 0x7a, 0xe2, 0x4e, 0xd0, 0xdf, 0x97, 0x6e, 0xd5, 0x4c, 0x3e, 0x9c, 0x0d, 0x81, 0xfa, 0x50, + 0xa9, 0xfa, 0xbd, 0x53, 0x97, 0x9c, 0x13, 0xbf, 0x7d, 0x71, 0x62, 0x37, 0xc5, 0xd5, 0x10, 0x36, + 0x72, 0x97, 0x35, 0xf2, 0x7e, 0x34, 0x88, 0x1c, 0x66, 0x3c, 0x02, 0x0c, 0xfd, 0x16, 0xf4, 0x7a, + 0xb3, 0x7a, 0x5a, 0x73, 0x4d, 0xcb, 0xae, 0x3a, 0xb6, 0x37, 0xec, 0x47, 0x3a, 0xe7, 0x1e, 0x6b, + 0x68, 0x93, 0x37, 0x94, 0xc5, 0x86, 0x33, 0x01, 0xd0, 0x00, 0x36, 0xea, 0x6d, 0x72, 0x46, 0xdc, + 0x2e, 0x37, 0xac, 0xde, 0x9a, 0x3d, 0xab, 0x63, 0xfa, 0xe1, 0x26, 0xdc, 0x66, 0x2d, 0xdc, 0x11, + 0x2d, 0xe4, 0xf1, 0xf2, 0x65, 0xc9, 0x07, 0x44, 0x5d, 0x58, 0x3b, 0x76, 0xec, 0xae, 0x53, 0x3b, + 0xa8, 0xf6, 0x2c, 0xb6, 0xbb, 0x7c, 0xcb, 0x0d, 0x9b, 0xbb, 0xcf, 0x9a, 0xbb, 0x2d, 0xf6, 0x7c, + 0x26, 0x23, 0x6f, 0x2b, 0x07, 0xca, 0x38, 0x84, 0x95, 0x94, 0xa1, 0xcf, 0x3d, 0x9d, 0x87, 0x50, + 0xe4, 0xea, 0xce, 0xd3, 0x0b, 0x5b, 0xe3, 0xdb, 0x33, 0x7b, 0xf3, 0x3b, 0x3c, 0x80, 0x24, 0xd4, + 0x60, 0xc8, 0x60, 0xfc, 0x71, 0x0d, 0x8a, 0xa1, 0xe4, 0x8f, 0xeb, 0x01, 0x2d, 0xc2, 0xe4, 0x63, + 0xd7, 0x75, 0x5c, 0xe6, 0xfa, 0x94, 0x71, 0x50, 0x40, 0x6f, 0x32, 0x3b, 0xce, 0xfd, 0x9b, 0x4a, + 0x96, 0x7f, 0x13, 0x70, 0xe1, 0xcc, 0x71, 0x9f, 0xc0, 0xa2, 0xec, 0xaa, 0x70, 0xd8, 0x49, 0x49, + 0xd3, 0xa8, 0x58, 0xb0, 0x52, 0x90, 0xde, 0x83, 0x91, 0xd7, 0xc2, 0xc1, 0xa6, 0xa4, 0x7b, 0x30, + 0x59, 0x8d, 0x53, 0x02, 0xd4, 0xaf, 0x88, 0xb9, 0x2d, 0x1c, 0x65, 0x5a, 0xba, 0x1c, 0x52, 0xf5, + 0x38, 0x2d, 0xc2, 0xad, 0xa8, 0xc8, 0x6b, 0xe1, 0x48, 0xc5, 0xa4, 0x15, 0x95, 0xe4, 0xc0, 0x2a, + 0x31, 0xee, 0x38, 0x85, 0xae, 0x0b, 0x07, 0x2b, 0x25, 0x1d, 0xa7, 0x04, 0x03, 0x56, 0x08, 0xd1, + 0x69, 0x97, 0x3d, 0x17, 0x0e, 0x06, 0x49, 0x13, 0x36, 0xc5, 0x82, 0x95, 0x82, 0xe8, 0x97, 0xd4, + 0xe7, 0x11, 0xae, 0x0d, 0xf7, 0x79, 0x56, 0x15, 0x3e, 0x0f, 0x07, 0x89, 0x31, 0xa3, 0x47, 0x69, + 0xa7, 0x47, 0x4f, 0x3b, 0x3d, 0x5c, 0x30, 0xe6, 0xf5, 0xbc, 0xc8, 0xf1, 0x7a, 0x6e, 0xe7, 0x78, + 0x3d, 0xb1, 0x69, 0x49, 0x3a, 0x29, 0x2f, 0x72, 0xdc, 0x9e, 0xdb, 0x39, 0x6e, 0x8f, 0x80, 0x54, + 0xf8, 0x3d, 0x8f, 0x33, 0xfc, 0x9e, 0x8d, 0x0c, 0xbf, 0x87, 0x43, 0x25, 0x1d, 0x9f, 0x0f, 0x93, + 0x8e, 0xcf, 0x72, 0xd2, 0xf1, 0xe1, 0x82, 0xa1, 0xe7, 0xf3, 0x79, 0xbe, 0xe7, 0x73, 0x27, 0xdf, + 0xf3, 0xe1, 0x68, 0x19, 0xae, 0xcf, 0x81, 0xda, 0xf5, 0x59, 0x57, 0xbb, 0x3e, 0x1c, 0x2b, 0xe1, + 0xfb, 0x3c, 0xcb, 0xf4, 0x7d, 0x36, 0x33, 0x7d, 0x1f, 0x71, 0x60, 0x53, 0xce, 0x4f, 0x6c, 0x3f, + 0x07, 0x5e, 0x0c, 0xdf, 0xcf, 0x8b, 0xca, 0xfd, 0x1c, 0x67, 0xc1, 0x4a, 0x41, 0x0e, 0x18, 0x73, + 0x64, 0x38, 0xe0, 0x52, 0x12, 0x30, 0xc5, 0x82, 0x95, 0x82, 0x54, 0x85, 0x66, 0x44, 0xb9, 0xb8, + 0x0f, 0x54, 0xc9, 0xf2, 0x81, 0x84, 0x0a, 0xcd, 0x0a, 0x92, 0xbd, 0x81, 0x95, 0x94, 0x23, 0xc3, + 0x91, 0x57, 0x24, 0xe4, 0x0c, 0x2e, 0x9c, 0x25, 0x8e, 0x30, 0x2c, 0x25, 0xfc, 0x19, 0x8e, 0xab, + 0x4b, 0xcb, 0xad, 0xe4, 0xc1, 0x6a, 0x51, 0xd4, 0x1e, 0xe9, 0x0b, 0xdd, 0x1b, 0xe9, 0x0b, 0xf1, + 0x16, 0xb2, 0x9d, 0xa1, 0x43, 0x58, 0x88, 0xf9, 0x36, 0xbc, 0xd3, 0x6b, 0x92, 0x6a, 0x49, 0xd5, + 0xe3, 0xb4, 0x08, 0x7a, 0x9e, 0xe5, 0x0f, 0x55, 0xb2, 0xfc, 0xa1, 0x40, 0x30, 0xcb, 0x21, 0x3a, + 0x81, 0x45, 0xd9, 0xb3, 0xe1, 0x5d, 0x5b, 0x97, 0x76, 0x95, 0x8a, 0x05, 0x2b, 0x05, 0x03, 0x8b, + 0x3a, 0x72, 0x6d, 0x38, 0xdc, 0x46, 0xc2, 0xa2, 0x4e, 0x32, 0x44, 0x16, 0x75, 0xb2, 0x86, 0x03, + 0x86, 0xde, 0x0d, 0x07, 0xac, 0x24, 0x01, 0x13, 0x0c, 0x31, 0xc0, 0x44, 0x0d, 0x32, 0x41, 0x4f, + 0x3b, 0x35, 0x1c, 0x76, 0x53, 0x3a, 0xee, 0x59, 0x6c, 0x1c, 0x3c, 0x13, 0x86, 0x5a, 0x8c, 0x19, + 0xde, 0x0c, 0x6f, 0x67, 0x4b, 0xd2, 0x78, 0xb9, 0xbc, 0xc2, 0x62, 0xcc, 0x65, 0x42, 0x6f, 0x60, + 0x29, 0xe1, 0xe0, 0xf0, 0x96, 0x6e, 0xcb, 0x07, 0x43, 0xc5, 0xc3, 0x5b, 0x50, 0x03, 0x20, 0x0c, + 0x37, 0x25, 0x3f, 0x87, 0xe3, 0x1a, 0xb2, 0xc5, 0x90, 0xe6, 0xe0, 0xa8, 0x2a, 0x61, 0x6a, 0x85, + 0x48, 0x0e, 0x0f, 0xc7, 0x7c, 0x4f, 0xc2, 0x54, 0x70, 0x60, 0x95, 0x18, 0x75, 0x75, 0x53, 0x5e, + 0x0e, 0x47, 0xbc, 0x23, 0x9d, 0x8d, 0x0c, 0x2e, 0xe1, 0xea, 0x66, 0x54, 0x23, 0x13, 0xd6, 0x54, + 0x8e, 0x0e, 0x6f, 0xe2, 0x7d, 0xe9, 0x2e, 0xce, 0x66, 0xc4, 0x39, 0x20, 0x41, 0x80, 0xc7, 0x0e, + 0x93, 0x70, 0x21, 0xf8, 0xdd, 0x44, 0x80, 0x27, 0xcd, 0x82, 0x95, 0x82, 0x68, 0x00, 0x9b, 0x99, + 0x2e, 0x13, 0xc7, 0xbe, 0x27, 0xc5, 0x79, 0x46, 0x70, 0xe3, 0x51, 0x70, 0xd4, 0xa3, 0x54, 0x78, + 0x50, 0xbc, 0xad, 0x6d, 0xc9, 0xa3, 0xcc, 0xe4, 0xc3, 0xd9, 0x10, 0xc8, 0x83, 0x4a, 0x96, 0xd3, + 0xc4, 0x1b, 0xb9, 0x2f, 0x79, 0x94, 0xf9, 0xcc, 0x7c, 0xcd, 0x47, 0x40, 0xa2, 0xdf, 0xc1, 0x2d, + 0xa5, 0xf7, 0xc4, 0x5b, 0x7c, 0xc0, 0x5a, 0x34, 0xf2, 0x3c, 0x31, 0xa9, 0xb9, 0x3c, 0x30, 0xa3, + 0xae, 0x4c, 0xad, 0xb0, 0x6c, 0x17, 0x4b, 0x53, 0xd7, 0x3b, 0x3c, 0xe9, 0x14, 0x96, 0xd1, 0x32, + 0x4c, 0x35, 0x99, 0x4b, 0xc6, 0x9c, 0xa3, 0x12, 0xe6, 0x25, 0xe3, 0x57, 0x6a, 0x1f, 0x06, 0x19, + 0x50, 0x36, 0x29, 0xbd, 0x39, 0x6c, 0x53, 0xbf, 0x87, 0xe1, 0x15, 0xb1, 0x44, 0x33, 0xea, 0xa9, + 0x9c, 0x0c, 0x75, 0xe8, 0x38, 0x12, 0x77, 0xe8, 0xc6, 0x71, 0x44, 0x88, 0xa7, 0xee, 0xc6, 0x98, + 0xb3, 0x17, 0x4b, 0xdd, 0xa5, 0x1d, 0x19, 0x1d, 0xa6, 0xe5, 0xd6, 0x45, 0xd1, 0x68, 0xa4, 0xe3, + 0x85, 0x48, 0x83, 0xf1, 0x6a, 0xbf, 0xc3, 0x13, 0x77, 0xf4, 0x93, 0x51, 0xce, 0xbb, 0xac, 0x25, + 0x4a, 0x39, 0xef, 0x32, 0x07, 0xf1, 0xca, 0x77, 0xcd, 0xd0, 0x41, 0xa4, 0x05, 0xe3, 0x9e, 0xe2, + 0xc2, 0x45, 0x08, 0x26, 0xe8, 0x37, 0xc7, 0x63, 0xdf, 0xc6, 0xa7, 0xa3, 0x22, 0x15, 0x74, 0x05, + 0x4e, 0x4d, 0xdf, 0x27, 0x2e, 0xf7, 0x84, 0x4b, 0x38, 0x2c, 0x1b, 0x1f, 0x8f, 0x3c, 0x67, 0xca, + 0x46, 0xff, 0xa3, 0x90, 0x1d, 0xb0, 0x48, 0x4f, 0xf7, 0x6c, 0x62, 0xba, 0x99, 0x8e, 0xaa, 0xd7, + 0xc4, 0x74, 0xf3, 0x22, 0xad, 0x79, 0xea, 0x9c, 0x3d, 0x37, 0xfb, 0x84, 0x6f, 0x07, 0x51, 0xa4, + 0x53, 0xf4, 0xd4, 0x39, 0xab, 0xd7, 0x98, 0x6f, 0x3c, 0x81, 0x83, 0x02, 0xda, 0x86, 0xf9, 0xc0, + 0x98, 0x66, 0x19, 0xeb, 0x13, 0xbb, 0x77, 0xcd, 0x9c, 0xdc, 0x22, 0x4e, 0x92, 0xd1, 0x5d, 0x98, + 0xc3, 0xc4, 0x26, 0x5f, 0x45, 0x8c, 0x53, 0x8c, 0x31, 0x41, 0x35, 0x3e, 0xce, 0xd1, 0x01, 0x39, + 0x2b, 0xff, 0x26, 0x9d, 0xc4, 0x53, 0xac, 0xfc, 0x22, 0x4c, 0x52, 0x06, 0x8f, 0xaf, 0x7d, 0x50, + 0xa0, 0x93, 0x15, 0xaa, 0x53, 0x36, 0xec, 0x71, 0x1c, 0x11, 0xe8, 0x2e, 0x48, 0xfb, 0xc0, 0xaa, + 0x05, 0x59, 0x54, 0xa5, 0x00, 0x8d, 0xbf, 0x14, 0xa0, 0x28, 0x68, 0xd1, 0xc4, 0x77, 0x78, 0x50, + 0x43, 0x14, 0x29, 0xe0, 0x33, 0x72, 0x1d, 0x24, 0xe6, 0xcb, 0x98, 0x7d, 0xa3, 0x07, 0x81, 0xe4, + 0xb1, 0xd3, 0x09, 0x56, 0x63, 0x6e, 0x6f, 0x6e, 0x87, 0xbd, 0xb8, 0x11, 0x54, 0x1c, 0xd6, 0xa3, + 0x2d, 0x98, 0xb1, 0x3c, 0x6c, 0xda, 0x5d, 0xe6, 0xd0, 0xb0, 0x45, 0x2a, 0xe2, 0x38, 0x09, 0xdd, + 0x83, 0xe9, 0x27, 0x4e, 0xaf, 0x43, 0x5c, 0x4f, 0x9f, 0x64, 0xb1, 0x98, 0xd9, 0x00, 0xec, 0xb5, + 0x69, 0x51, 0x4f, 0x1a, 0x8b, 0x5a, 0xca, 0x48, 0x69, 0x94, 0x71, 0x4a, 0xc9, 0xc8, 0x6b, 0x8d, + 0x2f, 0x94, 0x81, 0x00, 0x3a, 0x94, 0xaa, 0x5d, 0x17, 0xf3, 0xce, 0xbe, 0xd1, 0xcf, 0xa1, 0x2c, + 0xf8, 0x1a, 0x96, 0xe7, 0xb3, 0x61, 0xce, 0xec, 0xcd, 0x73, 0xad, 0x17, 0x42, 0x48, 0x4c, 0xc6, + 0x4d, 0x45, 0x22, 0xd4, 0xb8, 0x80, 0x99, 0xd6, 0x95, 0xfd, 0x6e, 0x33, 0x8a, 0x9d, 0xaf, 0xc2, + 0x19, 0xa5, 0xdf, 0xe8, 0x21, 0x4c, 0x9f, 0x0c, 0x7c, 0x16, 0x8f, 0x0a, 0xb2, 0xe0, 0x0b, 0xd1, + 0x84, 0xf2, 0x0a, 0x2c, 0x38, 0x8c, 0xff, 0x2c, 0xc0, 0x34, 0x6f, 0x1c, 0x7d, 0x06, 0xc5, 0xaa, + 0x4b, 0x4c, 0x9f, 0xec, 0xfb, 0xfc, 0xe5, 0xcb, 0xda, 0x4e, 0xf0, 0x10, 0x69, 0x47, 0x3c, 0x44, + 0x8a, 0xbd, 0x7f, 0x29, 0x52, 0x4d, 0xfd, 0xfb, 0xff, 0xd9, 0x2c, 0xe0, 0x50, 0x0a, 0x6d, 0xc1, + 0x04, 0x35, 0x8e, 0xd8, 0xce, 0x9b, 0xd9, 0x2b, 0xef, 0xf8, 0x57, 0xf6, 0x4e, 0xeb, 0xca, 0xa6, + 0x34, 0xcc, 0x6a, 0xe8, 0x50, 0x5e, 0x7a, 0xc4, 0x6d, 0x5d, 0xd9, 0xfc, 0x75, 0x88, 0x28, 0xa2, + 0x0f, 0xa1, 0x44, 0xe7, 0x9c, 0xf6, 0xd2, 0xd3, 0x27, 0xd8, 0xd4, 0x21, 0x11, 0xb1, 0x89, 0xe6, + 0x02, 0x47, 0x4c, 0xf2, 0xf9, 0x9f, 0x4c, 0x9c, 0x7f, 0xe3, 0x73, 0x55, 0xcc, 0x45, 0xb9, 0x6e, + 0x1f, 0xb2, 0xd9, 0x4e, 0x2c, 0xdb, 0x5c, 0xd4, 0x36, 0x03, 0x88, 0xb3, 0x18, 0x4b, 0xca, 0xac, + 0xb3, 0xf1, 0x2f, 0x05, 0x28, 0x85, 0x44, 0xaa, 0x0e, 0x9f, 0x3b, 0x1d, 0xd2, 0xba, 0x1e, 0x10, + 0xde, 0x5c, 0x58, 0xa6, 0x17, 0x12, 0xfd, 0xae, 0x77, 0xf8, 0x21, 0xe5, 0x25, 0x3a, 0x24, 0x06, + 0xc0, 0x84, 0x02, 0xe5, 0x14, 0x11, 0x68, 0xe7, 0x5f, 0x7a, 0xa4, 0xc3, 0xb5, 0x13, 0xfb, 0xa6, + 0xb4, 0x43, 0x97, 0x04, 0x61, 0xb7, 0x09, 0xcc, 0xbe, 0x69, 0xcb, 0x4f, 0x2c, 0x1f, 0x9b, 0xbe, + 0xe5, 0x30, 0x05, 0x34, 0x86, 0xc3, 0xb2, 0xf1, 0x5c, 0x1d, 0x3f, 0x42, 0x8f, 0x60, 0x36, 0x24, + 0xb2, 0x69, 0x08, 0x62, 0x99, 0x61, 0xc8, 0x31, 0x14, 0x90, 0xd9, 0x8c, 0x1e, 0xac, 0xe7, 0xa5, + 0x60, 0xe9, 0x82, 0x1f, 0xb9, 0xce, 0x70, 0x10, 0xaa, 0x68, 0x51, 0xcc, 0x57, 0xd0, 0xe2, 0xa6, + 0x1c, 0x97, 0x6f, 0xca, 0x8f, 0x61, 0x23, 0x37, 0xec, 0x21, 0xbf, 0x3b, 0x99, 0x14, 0xef, 0x4e, + 0x9e, 0xb2, 0x41, 0xa7, 0x92, 0xba, 0xdf, 0xa7, 0x73, 0xc6, 0x43, 0x58, 0x52, 0x46, 0x49, 0xe8, + 0x4a, 0xb0, 0x88, 0x0a, 0xdf, 0x5a, 0xf4, 0xdb, 0x68, 0xc2, 0x4a, 0x46, 0x1e, 0x18, 0x55, 0x00, + 0x6a, 0xa6, 0x6f, 0x9e, 0x99, 0x1e, 0x09, 0xc3, 0xbf, 0x31, 0x4a, 0x4e, 0x0f, 0x3e, 0x02, 0x3d, + 0x2b, 0xc0, 0x92, 0x73, 0x79, 0x1c, 0x42, 0x91, 0xad, 0xdc, 0x33, 0x72, 0x4d, 0xbb, 0x7a, 0x6a, + 0xfa, 0x17, 0xa2, 0xab, 0xf4, 0x9b, 0x6e, 0xc9, 0x93, 0xf3, 0x73, 0x8f, 0x04, 0xef, 0xfe, 0xc6, + 0x31, 0x2f, 0xa1, 0x39, 0x18, 0x6b, 0x7e, 0xcd, 0x6f, 0x8c, 0xb1, 0xe6, 0xd7, 0xc6, 0x23, 0xbe, + 0x45, 0x99, 0xf6, 0xbe, 0x0f, 0x13, 0x97, 0x54, 0xa3, 0x17, 0x24, 0x55, 0x27, 0xea, 0xb9, 0x35, + 0xc7, 0x58, 0x8c, 0x16, 0xbd, 0x45, 0xd9, 0xd0, 0xc3, 0x6e, 0x2c, 0xc2, 0x64, 0xdd, 0xee, 0x90, + 0x2b, 0xb1, 0x58, 0xac, 0x80, 0x1e, 0x46, 0x1d, 0xe5, 0x8a, 0x24, 0x89, 0x8b, 0x43, 0x06, 0xe3, + 0xb5, 0x32, 0x77, 0x8e, 0x3e, 0x4b, 0x35, 0xc6, 0xbb, 0x18, 0x06, 0xdf, 0xe4, 0x5a, 0x9c, 0x64, + 0x37, 0x4e, 0x60, 0x41, 0x4c, 0x6a, 0x88, 0x9e, 0xd1, 0x61, 0x0d, 0xc6, 0x9f, 0x58, 0xe2, 0xb9, + 0x24, 0xfd, 0xa4, 0xf3, 0x4b, 0xf9, 0xb9, 0xa5, 0xc5, 0xbe, 0x8d, 0x2f, 0xd4, 0x81, 0x2e, 0x74, + 0xa8, 0x68, 0x88, 0x77, 0x56, 0x8f, 0xc2, 0x0a, 0x72, 0x3d, 0x4e, 0x8b, 0x18, 0x58, 0x99, 0xf7, + 0x47, 0xbf, 0x86, 0x72, 0x48, 0x0b, 0xa6, 0x21, 0x88, 0xa8, 0x47, 0xaf, 0x55, 0xe3, 0xd5, 0x58, + 0x62, 0xe6, 0xe7, 0x26, 0x1d, 0x12, 0xdb, 0x83, 0x52, 0x48, 0x0c, 0x1f, 0x49, 0x2a, 0x10, 0x71, + 0xc4, 0x66, 0x34, 0x61, 0x86, 0xbf, 0x1c, 0x6c, 0xfa, 0x7d, 0x36, 0x45, 0xcc, 0x02, 0xe3, 0x5b, + 0x90, 0x99, 0x5f, 0x1a, 0x8c, 0x37, 0x5f, 0x34, 0x84, 0xcd, 0xda, 0x7c, 0xd1, 0xa0, 0x87, 0xe4, + 0xd4, 0x74, 0xcd, 0x3e, 0x55, 0x7f, 0x1e, 0x9f, 0xce, 0x18, 0xc5, 0xb8, 0x00, 0x8d, 0x07, 0xab, + 0x5a, 0xa4, 0x3f, 0x08, 0x22, 0x8a, 0x6b, 0x50, 0x14, 0xc7, 0x48, 0xe8, 0x5d, 0x51, 0xa6, 0x0b, + 0xb8, 0xdf, 0xb3, 0xcc, 0xd0, 0x36, 0x62, 0x05, 0x6a, 0xee, 0x9f, 0x5e, 0x5c, 0x7b, 0x56, 0xdb, + 0xec, 0xc5, 0xac, 0x42, 0x89, 0x66, 0xfc, 0x5b, 0x21, 0xeb, 0xc9, 0x02, 0x3d, 0x39, 0x94, 0x14, + 0x2a, 0x11, 0x5e, 0x42, 0x1f, 0x85, 0x2f, 0x3e, 0x83, 0xf4, 0xcd, 0xba, 0x3a, 0x9a, 0x17, 0xf0, + 0x88, 0xf7, 0xa0, 0xe8, 0x33, 0xb8, 0x15, 0x8e, 0x25, 0xe0, 0xb4, 0x1c, 0xbb, 0x39, 0x1c, 0x0c, + 0x1c, 0xd7, 0xe7, 0xba, 0xbf, 0x88, 0xf3, 0x58, 0x9e, 0x4e, 0x14, 0xc7, 0xb5, 0x09, 0xe3, 0xff, + 0xa6, 0x33, 0xa3, 0x8b, 0xf4, 0x4c, 0xd7, 0x0e, 0xf8, 0xe4, 0x8c, 0xd5, 0x0e, 0xd0, 0x23, 0x28, + 0xc7, 0xd6, 0xc6, 0xe3, 0x57, 0xa0, 0xb8, 0x7e, 0x63, 0x55, 0x58, 0xe2, 0x43, 0x0f, 0x40, 0x6b, + 0x98, 0x9e, 0xbf, 0x7f, 0x7e, 0x4e, 0xda, 0x3e, 0xe9, 0x30, 0x53, 0x24, 0xd0, 0x14, 0x29, 0x3a, + 0xfa, 0x14, 0xe6, 0xe8, 0x55, 0xdf, 0x20, 0x6f, 0x49, 0x2f, 0x7e, 0xc9, 0x2f, 0x8a, 0x98, 0x71, + 0xbc, 0x12, 0x27, 0x78, 0x51, 0x0d, 0x36, 0x64, 0x06, 0xd2, 0x23, 0xa6, 0x47, 0xa2, 0x79, 0x09, + 0x2c, 0xf2, 0x7c, 0x26, 0x74, 0x08, 0xf3, 0x94, 0xa1, 0x46, 0xce, 0x2d, 0x9b, 0x74, 0x5e, 0x99, + 0xa1, 0xf5, 0x97, 0x58, 0x1a, 0x99, 0x09, 0x27, 0x85, 0xd0, 0x2f, 0x60, 0x25, 0x41, 0x7a, 0x7c, + 0xc5, 0xfb, 0x31, 0xcd, 0xfa, 0x91, 0x55, 0x4d, 0x7b, 0xd0, 0xbc, 0xf6, 0x7c, 0xd2, 0x7f, 0x65, + 0xba, 0x16, 0x5d, 0x3f, 0x4f, 0x2f, 0xaa, 0x7a, 0x20, 0x33, 0xe1, 0xa4, 0x10, 0xed, 0x41, 0x82, + 0x14, 0xf6, 0xa0, 0x14, 0xf4, 0x20, 0xa3, 0x1a, 0x7d, 0x0a, 0xab, 0x89, 0xce, 0x61, 0x32, 0xe8, + 0x99, 0xd7, 0x2c, 0x8c, 0x0f, 0x4c, 0x36, 0x9b, 0x81, 0x4a, 0x27, 0x80, 0x63, 0xd2, 0x33, 0x81, + 0x74, 0x26, 0x03, 0x7a, 0x02, 0x9b, 0x89, 0xca, 0xa6, 0x6d, 0x0e, 0xbc, 0x0b, 0xc7, 0x6f, 0x39, + 0x4e, 0xc3, 0x74, 0xbb, 0x84, 0xa5, 0x91, 0x8a, 0x78, 0x14, 0x1b, 0x45, 0x4a, 0x74, 0x32, 0x85, + 0x34, 0x1b, 0x20, 0x8d, 0x60, 0xa3, 0xe7, 0x4d, 0xbc, 0x68, 0xa6, 0x9b, 0xba, 0xe1, 0xd8, 0x5d, + 0xaa, 0x2e, 0xaa, 0x17, 0xa4, 0x7d, 0x49, 0x3a, 0x2c, 0x85, 0x54, 0xc4, 0x79, 0x2c, 0xd4, 0x68, + 0x3b, 0x74, 0x86, 0x76, 0xb0, 0xfd, 0xe7, 0x83, 0x3c, 0x6e, 0x48, 0x40, 0x9f, 0x00, 0x84, 0x87, + 0xd5, 0xd3, 0x35, 0xb6, 0xd8, 0x2b, 0xf2, 0x62, 0x87, 0xf5, 0x38, 0xc6, 0x8a, 0x1e, 0xc1, 0x72, + 0x58, 0xa2, 0x6a, 0x94, 0x84, 0x2b, 0xbc, 0xc0, 0xfa, 0x94, 0x51, 0x6b, 0xfc, 0xd7, 0xa4, 0xfa, + 0x15, 0x54, 0xa6, 0x9e, 0x0a, 0xb4, 0xc1, 0x58, 0xa8, 0x0d, 0xb6, 0x60, 0xa6, 0x49, 0xfc, 0x57, + 0xa6, 0x1b, 0x28, 0x83, 0x71, 0xe6, 0xca, 0xc7, 0x49, 0x29, 0x7d, 0x31, 0xf1, 0x03, 0xf4, 0xc5, + 0xe4, 0x3b, 0xeb, 0x8b, 0xa9, 0xef, 0xa0, 0x2f, 0x14, 0x27, 0x7d, 0xfa, 0x47, 0x3e, 0xe9, 0xc5, + 0xef, 0x7c, 0xd2, 0x4b, 0x3f, 0xf2, 0x49, 0x87, 0x1f, 0x70, 0xd2, 0x67, 0x7e, 0xd0, 0x49, 0x2f, + 0x8f, 0x3a, 0xe9, 0xd2, 0x99, 0x98, 0xcd, 0x3f, 0x13, 0x73, 0xef, 0x7c, 0x26, 0x8c, 0x3f, 0x16, + 0xc2, 0xc4, 0x96, 0xdc, 0x73, 0xa5, 0x3d, 0xb1, 0x05, 0x93, 0xaf, 0xcc, 0xde, 0x90, 0x70, 0x33, + 0x12, 0x76, 0xd8, 0x2f, 0x72, 0x1e, 0x5f, 0x0d, 0x5c, 0x1c, 0x54, 0x30, 0x8b, 0xe3, 0xcb, 0x1e, + 0xbf, 0xf0, 0xe9, 0x27, 0x33, 0xf1, 0xbc, 0x03, 0xcb, 0xe6, 0x17, 0x6d, 0x50, 0xa0, 0x1b, 0x97, + 0x6f, 0x64, 0x66, 0x7c, 0x3c, 0xa3, 0x76, 0x7e, 0xe0, 0x71, 0xa6, 0xe8, 0xa8, 0x02, 0x13, 0xcc, + 0x7d, 0x9b, 0x8a, 0x37, 0x4a, 0x29, 0x98, 0xd1, 0x8d, 0x17, 0x30, 0x2b, 0x6d, 0x56, 0x65, 0xd7, + 0x43, 0x3f, 0x26, 0xb0, 0xfd, 0x83, 0x02, 0xfb, 0xd9, 0xce, 0x95, 0x5d, 0xaf, 0x05, 0x87, 0xb2, + 0x8c, 0x79, 0xc9, 0xf8, 0x9b, 0x8c, 0x74, 0x5f, 0x8e, 0x3b, 0xf0, 0x6b, 0xd8, 0x1c, 0xf1, 0x64, + 0x31, 0xee, 0x81, 0x14, 0x64, 0x0f, 0xc4, 0x80, 0xad, 0x51, 0x39, 0x3e, 0x63, 0x9b, 0xbd, 0x1c, + 0x55, 0xbc, 0x39, 0xa4, 0xfa, 0xa6, 0xfa, 0x5c, 0x58, 0x1f, 0xd5, 0xe7, 0xfc, 0x67, 0x04, 0xaa, + 0x6c, 0x5c, 0xc6, 0xcf, 0x08, 0x3e, 0x50, 0xbe, 0x4e, 0xcc, 0xd2, 0x6f, 0xc6, 0xa9, 0x3a, 0x77, + 0x97, 0x3d, 0x39, 0xd4, 0xec, 0xdc, 0x1f, 0xfa, 0x17, 0x4d, 0xdf, 0xb5, 0xec, 0x20, 0x86, 0x5a, + 0xc6, 0x31, 0x8a, 0xb1, 0xab, 0x78, 0xd0, 0x48, 0xed, 0x4e, 0x41, 0x12, 0x3f, 0xb7, 0x10, 0x65, + 0xe3, 0x43, 0x55, 0xb6, 0x2f, 0x57, 0xe2, 0x97, 0x8a, 0x57, 0x8e, 0xe8, 0x0e, 0xcc, 0x0a, 0xd2, + 0xc1, 0xb5, 0x4f, 0x3c, 0x3e, 0x2d, 0x32, 0xd1, 0xf8, 0x95, 0x2a, 0x13, 0xf8, 0x8e, 0xb2, 0x17, + 0x99, 0xcf, 0x21, 0xd1, 0x2e, 0xdf, 0xd7, 0x05, 0x66, 0xcc, 0xde, 0xca, 0xc8, 0xc9, 0x45, 0x1b, + 0x3d, 0x0c, 0x66, 0x34, 0xad, 0xaf, 0x09, 0x77, 0x2a, 0x23, 0x82, 0x71, 0x95, 0x9d, 0x5e, 0x94, + 0x25, 0x0b, 0x09, 0x49, 0x3a, 0x12, 0x56, 0xa8, 0x9a, 0x03, 0xb3, 0x6d, 0xf9, 0xd7, 0x1c, 0x5b, + 0x26, 0xd2, 0xd5, 0x3d, 0x26, 0x9e, 0x67, 0x76, 0xc3, 0x28, 0x2f, 0x2f, 0x1a, 0x27, 0xf9, 0x4f, + 0x32, 0xbf, 0xf3, 0x40, 0x8d, 0xbf, 0x1f, 0x91, 0xc6, 0xfc, 0x89, 0xc7, 0xf3, 0x91, 0xfa, 0xcd, + 0x67, 0x7e, 0xab, 0xc6, 0xdf, 0x66, 0x64, 0x42, 0xd3, 0xdd, 0x29, 0x28, 0xba, 0x63, 0xfc, 0x7f, + 0x61, 0xc4, 0x6b, 0xbf, 0x11, 0xe1, 0x79, 0xd6, 0x39, 0xdf, 0xec, 0x39, 0xdd, 0x30, 0xc0, 0x11, + 0x11, 0x68, 0x2d, 0x55, 0x83, 0x2c, 0x0b, 0x28, 0xe2, 0x60, 0x21, 0x81, 0x6a, 0x85, 0xe0, 0xd5, + 0x49, 0xf0, 0x93, 0xaa, 0xa0, 0x40, 0x0f, 0xad, 0x30, 0xee, 0x78, 0x3c, 0x70, 0x1c, 0xc7, 0x28, + 0x68, 0x2f, 0x9a, 0xa6, 0x86, 0xd3, 0x66, 0x3e, 0xd3, 0x13, 0xd3, 0xbb, 0x60, 0x7a, 0xba, 0x84, + 0x95, 0x75, 0xf4, 0x84, 0x06, 0x6f, 0x96, 0xea, 0x35, 0x66, 0xd9, 0x97, 0x70, 0x58, 0x36, 0x9e, + 0x8c, 0x4a, 0xb4, 0x05, 0xe9, 0x80, 0xbe, 0xf3, 0x96, 0x74, 0x1e, 0xdb, 0xbe, 0x6b, 0x85, 0x67, + 0x2e, 0x41, 0x35, 0x76, 0x54, 0xaf, 0x6b, 0xe9, 0x82, 0x73, 0x0a, 0x57, 0x0e, 0xa2, 0x68, 0xec, + 0x2a, 0x53, 0xcd, 0x39, 0x02, 0x0d, 0xd5, 0x6b, 0x5b, 0xaa, 0x2f, 0xf9, 0xf3, 0x42, 0xfe, 0x83, + 0x2f, 0xfe, 0x92, 0x50, 0x38, 0xdd, 0x84, 0x85, 0xc7, 0x03, 0xbb, 0x30, 0x46, 0x31, 0xee, 0x2b, + 0xb3, 0xd2, 0xca, 0x74, 0xc1, 0x83, 0xac, 0xb7, 0xb9, 0xe9, 0xbc, 0x85, 0xf1, 0x41, 0x66, 0x7a, + 0x5a, 0x09, 0xdd, 0xcf, 0x79, 0x99, 0x8b, 0xb6, 0x61, 0x9e, 0xff, 0x22, 0x33, 0xcc, 0x79, 0x04, + 0x57, 0x58, 0x92, 0x4c, 0xd7, 0xe8, 0xb5, 0x6b, 0xf9, 0x11, 0x04, 0xdf, 0x8c, 0x09, 0xaa, 0x61, + 0xe7, 0x25, 0xb7, 0x7f, 0x82, 0xf6, 0x5e, 0xa9, 0x33, 0xdd, 0xe8, 0xef, 0xa0, 0x1c, 0xa7, 0xbf, + 0xc3, 0x4f, 0x54, 0x25, 0x7e, 0xe3, 0x1f, 0x0a, 0x79, 0x6f, 0x66, 0x47, 0x1c, 0x5a, 0x03, 0xca, + 0xf4, 0x4e, 0x25, 0x2c, 0x52, 0x11, 0x9e, 0x5b, 0x89, 0x46, 0x4d, 0x25, 0x1e, 0xad, 0x7d, 0x7c, + 0xd5, 0xee, 0x0d, 0x3d, 0xeb, 0x2d, 0xe1, 0x51, 0xdc, 0x14, 0xdd, 0xf8, 0x24, 0x37, 0x6d, 0x9c, + 0x63, 0xbd, 0xfc, 0x6b, 0x64, 0x07, 0xca, 0x36, 0xe8, 0xf7, 0xb4, 0x03, 0xb7, 0x61, 0xfe, 0x39, + 0xb9, 0xf2, 0x5b, 0xae, 0x69, 0x7b, 0x66, 0x3b, 0xfc, 0xd9, 0x67, 0x11, 0x27, 0xc9, 0x68, 0x07, + 0xca, 0x78, 0x68, 0xd3, 0xd9, 0x0e, 0x20, 0x27, 0x52, 0x90, 0x52, 0xfd, 0x83, 0x3f, 0x4c, 0xc5, + 0xde, 0xf1, 0xa2, 0x12, 0xff, 0x9d, 0xb6, 0x76, 0x03, 0xdd, 0x84, 0xf9, 0xc4, 0xd3, 0x5a, 0xad, + 0x80, 0x34, 0x28, 0xc7, 0x13, 0xd2, 0xda, 0x18, 0x2a, 0x43, 0x51, 0xe4, 0x86, 0xb5, 0x71, 0x34, + 0x0b, 0xa5, 0x30, 0x4f, 0xa7, 0x4d, 0xa0, 0x79, 0x98, 0x89, 0x25, 0xa7, 0xb4, 0x49, 0x34, 0x07, + 0x10, 0x25, 0x3d, 0xb4, 0x29, 0x8a, 0x17, 0x8f, 0xf6, 0x6b, 0xd3, 0x94, 0x23, 0x7a, 0xc1, 0xa9, + 0x15, 0x29, 0x62, 0xf8, 0x30, 0x53, 0x2b, 0xa1, 0x65, 0xd5, 0xd3, 0x4c, 0x0d, 0x28, 0x3d, 0xfd, + 0x44, 0x52, 0x9b, 0x41, 0x28, 0xf9, 0x48, 0x52, 0x2b, 0xa3, 0x99, 0xf0, 0xc5, 0xa3, 0x36, 0x8b, + 0x56, 0x33, 0x1e, 0x33, 0x6a, 0x73, 0x68, 0x21, 0xf1, 0x16, 0x51, 0x9b, 0x47, 0x8b, 0xe9, 0xa7, + 0x85, 0x9a, 0x16, 0x1f, 0x05, 0xf5, 0xbb, 0xb5, 0x05, 0x4e, 0x09, 0x83, 0x8b, 0x1a, 0xa2, 0xd3, + 0x99, 0x78, 0x66, 0xa7, 0xdd, 0xa4, 0xc4, 0x44, 0x08, 0x4c, 0x5b, 0xa4, 0xcd, 0x4a, 0xc6, 0xb2, + 0xb6, 0x84, 0xd6, 0xb3, 0x9f, 0xb6, 0x69, 0xcb, 0x74, 0x8a, 0xc2, 0x14, 0xb9, 0xb6, 0xc2, 0x5b, + 0x8a, 0x9b, 0xab, 0x9a, 0x4e, 0x3b, 0x14, 0xb7, 0x31, 0xb5, 0x55, 0xb6, 0x14, 0x27, 0xc7, 0xfb, + 0x6f, 0x4e, 0xf1, 0x49, 0xb5, 0xa9, 0xad, 0xf1, 0xf2, 0xe3, 0xe3, 0x46, 0xfd, 0xb8, 0xde, 0xd2, + 0x6e, 0xd1, 0xa1, 0x26, 0x8d, 0x06, 0x6d, 0x9d, 0x4e, 0x97, 0xd2, 0x94, 0xd0, 0x36, 0x58, 0xbf, + 0xe3, 0x17, 0xb6, 0x56, 0x61, 0xeb, 0x7f, 0x12, 0x5e, 0x04, 0xda, 0x26, 0x25, 0xc4, 0x54, 0xb3, + 0xb6, 0x45, 0x3b, 0x9b, 0x50, 0xaa, 0xda, 0x6d, 0xba, 0x98, 0x69, 0x5d, 0xa6, 0x19, 0x74, 0x10, + 0x71, 0x5d, 0xa1, 0xbd, 0x87, 0x6e, 0x31, 0x9d, 0xac, 0x4a, 0xdb, 0x6b, 0x77, 0xd0, 0x12, 0x2c, + 0xa4, 0xb2, 0xd8, 0xda, 0xfb, 0x68, 0x0d, 0x96, 0xd5, 0xf7, 0xa2, 0x76, 0x17, 0xad, 0xc0, 0x4d, + 0xc5, 0x81, 0xd7, 0xee, 0xd1, 0xa6, 0xa3, 0x6d, 0xf9, 0x6a, 0x4f, 0xdb, 0x7e, 0xf0, 0x4f, 0x05, + 0xc9, 0xa9, 0x89, 0x22, 0xa5, 0x74, 0x8e, 0x12, 0x15, 0x81, 0xcb, 0xab, 0xdd, 0x40, 0x0f, 0xe1, + 0x5e, 0xa2, 0xaa, 0x79, 0x69, 0x0d, 0x54, 0x41, 0x41, 0xad, 0x80, 0x3e, 0x80, 0xfb, 0x49, 0x1c, + 0x9b, 0x4e, 0x93, 0x92, 0x7d, 0xec, 0xc1, 0xcf, 0x60, 0x51, 0x65, 0x03, 0x22, 0xa0, 0xd7, 0x68, + 0xdf, 0x61, 0x87, 0xba, 0x08, 0x13, 0x35, 0xcb, 0xbb, 0xd4, 0x0a, 0x07, 0x8d, 0x3f, 0x7d, 0x53, + 0x29, 0xfc, 0xf9, 0x9b, 0x4a, 0xe1, 0x7f, 0xbf, 0xa9, 0xdc, 0xf8, 0xfd, 0xb7, 0x95, 0x1b, 0xff, + 0xfc, 0x6d, 0xa5, 0xf0, 0xe7, 0x6f, 0x2b, 0x37, 0xfe, 0xf2, 0x6d, 0xe5, 0xc6, 0xe7, 0x3b, 0xb1, + 0xff, 0x07, 0xd1, 0x37, 0x7d, 0xd7, 0xba, 0x72, 0x5c, 0xab, 0x6b, 0xd9, 0xa2, 0x60, 0x93, 0xdd, + 0xc1, 0x65, 0x77, 0x77, 0x70, 0xb6, 0xcb, 0x8c, 0xcf, 0xb3, 0x29, 0x96, 0x80, 0xfd, 0xf9, 0x5f, + 0x03, 0x00, 0x00, 0xff, 0xff, 0xba, 0x5c, 0xd0, 0xce, 0xaf, 0x42, 0x00, 0x00, } func (m *QueryRequest) Marshal() (dAtA []byte, err error) { @@ -6544,6 +6602,26 @@ func (m *GetProtocolVersionResponse) MarshalToSizedBuffer(dAtA []byte) (int, err _ = i var l int _ = l + if m.DDLVisibilityActivationFenced { + i-- + if m.DDLVisibilityActivationFenced { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x18 + } + if m.DDLVisibilityActivationPrepared { + i-- + if m.DDLVisibilityActivationPrepared { + dAtA[i] = 1 + } else { + dAtA[i] = 0 + } + i-- + dAtA[i] = 0x10 + } if m.Version != 0 { i = encodeVarintQuery(dAtA, i, uint64(m.Version)) i-- @@ -6572,6 +6650,27 @@ func (m *SetProtocolVersionRequest) MarshalToSizedBuffer(dAtA []byte) (int, erro _ = i var l int _ = l + if len(m.DDLVisibilityTargetQueryAddress) > 0 { + i -= len(m.DDLVisibilityTargetQueryAddress) + copy(dAtA[i:], m.DDLVisibilityTargetQueryAddress) + i = encodeVarintQuery(dAtA, i, uint64(len(m.DDLVisibilityTargetQueryAddress))) + i-- + dAtA[i] = 0x22 + } + if m.DDLVisibilityTargetGeneration != 0 { + i = encodeVarintQuery(dAtA, i, uint64(m.DDLVisibilityTargetGeneration)) + i-- + dAtA[i] = 0x18 + } + if len(m.DDLVisibilityActivationTargets) > 0 { + for iNdEx := len(m.DDLVisibilityActivationTargets) - 1; iNdEx >= 0; iNdEx-- { + i -= len(m.DDLVisibilityActivationTargets[iNdEx]) + copy(dAtA[i:], m.DDLVisibilityActivationTargets[iNdEx]) + i = encodeVarintQuery(dAtA, i, uint64(len(m.DDLVisibilityActivationTargets[iNdEx]))) + i-- + dAtA[i] = 0x12 + } + } if m.Version != 0 { i = encodeVarintQuery(dAtA, i, uint64(m.Version)) i-- @@ -10996,6 +11095,12 @@ func (m *GetProtocolVersionResponse) ProtoSize() (n int) { if m.Version != 0 { n += 1 + sovQuery(uint64(m.Version)) } + if m.DDLVisibilityActivationPrepared { + n += 2 + } + if m.DDLVisibilityActivationFenced { + n += 2 + } return n } @@ -11008,6 +11113,19 @@ func (m *SetProtocolVersionRequest) ProtoSize() (n int) { if m.Version != 0 { n += 1 + sovQuery(uint64(m.Version)) } + if len(m.DDLVisibilityActivationTargets) > 0 { + for _, s := range m.DDLVisibilityActivationTargets { + l = len(s) + n += 1 + l + sovQuery(uint64(l)) + } + } + if m.DDLVisibilityTargetGeneration != 0 { + n += 1 + sovQuery(uint64(m.DDLVisibilityTargetGeneration)) + } + l = len(m.DDLVisibilityTargetQueryAddress) + if l > 0 { + n += 1 + l + sovQuery(uint64(l)) + } return n } @@ -13407,6 +13525,46 @@ func (m *GetProtocolVersionResponse) Unmarshal(dAtA []byte) error { break } } + case 2: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationPrepared", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowQuery + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityActivationPrepared = bool(v != 0) + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationFenced", wireType) + } + var v int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowQuery + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + v |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + m.DDLVisibilityActivationFenced = bool(v != 0) default: iNdEx = preIndex skippy, err := skipQuery(dAtA[iNdEx:]) @@ -13476,6 +13634,89 @@ func (m *SetProtocolVersionRequest) Unmarshal(dAtA []byte) error { break } } + case 2: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityActivationTargets", wireType) + } + var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowQuery + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthQuery + } + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthQuery + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.DDLVisibilityActivationTargets = append(m.DDLVisibilityActivationTargets, string(dAtA[iNdEx:postIndex])) + iNdEx = postIndex + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityTargetGeneration", wireType) + } + m.DDLVisibilityTargetGeneration = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowQuery + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.DDLVisibilityTargetGeneration |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 4: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field DDLVisibilityTargetQueryAddress", wireType) + } + var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowQuery + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthQuery + } + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthQuery + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.DDLVisibilityTargetQueryAddress = string(dAtA[iNdEx:postIndex]) + iNdEx = postIndex default: iNdEx = preIndex skippy, err := skipQuery(dAtA[iNdEx:]) diff --git a/pkg/queryservice/client/query_client.go b/pkg/queryservice/client/query_client.go index 9196d8816f8f1..b4d4de28e8525 100644 --- a/pkg/queryservice/client/query_client.go +++ b/pkg/queryservice/client/query_client.go @@ -63,6 +63,7 @@ var methodVersions = map[pb.CmdMethod]int64{ pb.CmdMethod_ISCPDrainConsumer: defines.MORPCVersion4, pb.CmdMethod_IcebergCacheInvalidate: defines.MORPCVersion4, pb.CmdMethod_MongoDBClientRetire: defines.MORPCVersion5, + pb.CmdMethod_SyncCommitV2: defines.MORPCVersion44, } type queryClient struct { diff --git a/pkg/queryservice/client/query_client_test.go b/pkg/queryservice/client/query_client_test.go index b119e07d7ec9c..97cf7ee584be8 100644 --- a/pkg/queryservice/client/query_client_test.go +++ b/pkg/queryservice/client/query_client_test.go @@ -15,10 +15,12 @@ package client import ( + "context" "testing" "github.com/matrixorigin/matrixone/pkg/common/moerr" "github.com/matrixorigin/matrixone/pkg/common/morpc" + moruntime "github.com/matrixorigin/matrixone/pkg/common/runtime" "github.com/matrixorigin/matrixone/pkg/defines" "github.com/matrixorigin/matrixone/pkg/pb/query" "github.com/stretchr/testify/assert" @@ -34,6 +36,19 @@ func TestMongoDBClientRetireRequiresProtocolVersion5(t *testing.T) { assert.Equal(t, defines.MORPCVersion5, methodVersions[query.CmdMethod_MongoDBClientRetire]) } +func TestSyncCommitV2RequiresProtocolVersion38(t *testing.T) { + assert.Equal(t, defines.MORPCVersion44, methodVersions[query.CmdMethod_SyncCommitV2]) + + const serviceID = "sync-commit-v2-version-test" + rt := moruntime.DefaultRuntime() + moruntime.SetupServiceBasedRuntime(serviceID, rt) + req := &query.Request{CmdMethod: query.CmdMethod_SyncCommitV2} + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion35) + assert.Error(t, checkMethodVersion(context.Background(), serviceID, req)) + rt.SetGlobalVariables(moruntime.MOProtocolVersion, defines.MORPCVersion44) + assert.NoError(t, checkMethodVersion(context.Background(), serviceID, req)) +} + func TestNewCacheClient(t *testing.T) { ct := testCreateQueryClient(t) assert.NotNil(t, ct) diff --git a/pkg/sql/plan/function/ctl/cmd_rpc_version.go b/pkg/sql/plan/function/ctl/cmd_rpc_version.go index 8dcff5b268dac..04d13c8c1d107 100644 --- a/pkg/sql/plan/function/ctl/cmd_rpc_version.go +++ b/pkg/sql/plan/function/ctl/cmd_rpc_version.go @@ -16,6 +16,7 @@ package ctl import ( "context" + "errors" "fmt" "strconv" "strings" @@ -23,6 +24,7 @@ import ( "github.com/matrixorigin/matrixone/pkg/clusterservice" "github.com/matrixorigin/matrixone/pkg/common/moerr" + "github.com/matrixorigin/matrixone/pkg/defines" "github.com/matrixorigin/matrixone/pkg/pb/metadata" querypb "github.com/matrixorigin/matrixone/pkg/pb/query" qclient "github.com/matrixorigin/matrixone/pkg/queryservice/client" @@ -111,18 +113,90 @@ func handleSetProtocolVersion(proc *process.Process, } if service == cn && targets != nil { - versions := make([]string, 0, len(targets)) - for _, target := range targets { - resp, err := transferToCN(qt, target, version) - if err != nil { + if version >= defines.MORPCVersion44 { + const maxDDLVisibilityActivationTargets = 1024 + if len(targets) == 0 || len(targets) > maxDDLVisibilityActivationTargets { + return Result{}, moerr.NewInternalErrorNoCtxf( + "DDL visibility activation target count must be between 1 and %d", + maxDDLVisibilityActivationTargets) + } + seen := make(map[string]struct{}, len(targets)) + for _, target := range targets { + if target == "" { + return Result{}, moerr.NewInternalErrorNoCtx("DDL visibility activation target is empty") + } + if _, ok := seen[target]; ok { + return Result{}, moerr.NewInternalErrorNoCtxf( + "DDL visibility activation target %s is duplicated", target) + } + seen[target] = struct{}{} + } + } + if version >= defines.MORPCVersion44 { + if err := validateDDLVisibilityActivationMembership(qt, targets); err != nil { return Result{}, err } - if resp == nil { - return Result{}, moerr.NewInternalErrorNoCtx("no such cn service") + } + if version < defines.MORPCVersion44 { + versions := make([]string, 0, len(targets)) + for _, target := range targets { + resp, sendErr := transferToCN(qt, target, version, nil) + if sendErr != nil { + return Result{}, sendErr + } + if resp == nil || resp.SetProtocolVersion == nil { + if resp != nil { + qt.Release(resp) + } + return Result{}, moerr.NewInternalErrorNoCtx("no such cn service") + } + versions = append(versions, fmt.Sprintf("%s:%d", target, resp.SetProtocolVersion.Version)) + qt.Release(resp) } - versions = append(versions, fmt.Sprintf("%s:%d", target, resp.SetProtocolVersion.Version)) + return Result{Method: SetProtocolVersionMethod, Data: strings.Join(versions, ", ")}, nil } + // Live v44 activation is a distributed barrier. Dispatch every target + // concurrently so each CN can block local DDL producers before any CN + // waits for the complete Prepared set. + type targetResult struct { + index int + resp *querypb.Response + err error + } + results := make(chan targetResult, len(targets)) + for i, target := range targets { + go func(index int, serviceID string) { + resp, sendErr := transferToCN(qt, serviceID, version, targets) + results <- targetResult{index: index, resp: resp, err: sendErr} + }(i, target) + } + + versions := make([]string, len(targets)) + var resultErr error + for range targets { + result := <-results + if result.err != nil { + if result.resp != nil { + qt.Release(result.resp) + } + resultErr = errors.Join(resultErr, result.err) + continue + } + if result.resp == nil || result.resp.SetProtocolVersion == nil { + if result.resp != nil { + qt.Release(result.resp) + } + resultErr = errors.Join(resultErr, moerr.NewInternalErrorNoCtx("no such cn service")) + continue + } + versions[result.index] = fmt.Sprintf( + "%s:%d", targets[result.index], result.resp.SetProtocolVersion.Version) + qt.Release(result.resp) + } + if resultErr != nil { + return Result{}, resultErr + } return Result{ Method: SetProtocolVersionMethod, Data: strings.Join(versions, ", "), @@ -153,6 +227,50 @@ func checkProtocolParameter(param string) ([]string, int64, error) { return nil, version, nil } +func validateDDLVisibilityActivationMembership(qt qclient.QueryClient, targets []string) error { + cluster := clusterservice.GetMOCluster(qt.ServiceID()) + refresher, ok := cluster.(clusterservice.AuthoritativeRefresher) + if !ok { + return moerr.NewInternalErrorNoCtx( + "CN cluster service does not support authoritative DDL activation inventory") + } + ctx, cancel := context.WithTimeoutCause(context.Background(), time.Minute, moerr.CauseTransferToCN) + defer cancel() + if err := refresher.Refresh(ctx); err != nil { + return moerr.AttachCause(ctx, err) + } + authoritative := make(map[string]metadata.CNService) + if err := clusterservice.GetCNServiceRawWithContext( + ctx, cluster, clusterservice.NewSelector(), func(cn metadata.CNService) bool { + if cn.QueryAddress != "" && cn.ViewMetadataAdmissionGeneration != 0 { + authoritative[cn.ServiceID] = cn + } + return true + }); err != nil { + return moerr.AttachCause(ctx, err) + } + requested := make(map[string]struct{}, len(targets)) + for _, target := range targets { + requested[target] = struct{}{} + } + if len(requested) != len(authoritative) { + return moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "DDL visibility activation target set does not match authoritative CN membership: requested=%d authoritative=%d", + len(requested), len(authoritative))) + } + for serviceID, cn := range authoritative { + if _, ok := requested[serviceID]; !ok { + return moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "DDL visibility activation omits authoritative CN %s", serviceID)) + } + if !cn.DDLVisibilityBarrierReady { + return moerr.NewInvalidStateNoCtx(fmt.Sprintf( + "authoritative CN %s does not support the DDL visibility activation receiver", serviceID)) + } + } + return nil +} + func transferToTN(qt qclient.QueryClient, version int64) (Result, error) { var addr string var resp *querypb.Response @@ -187,23 +305,59 @@ func transferToTN(qt qclient.QueryClient, version int64) (Result, error) { }, nil } -func transferToCN(qt qclient.QueryClient, target string, version int64) (resp *querypb.Response, err error) { - clusterservice.GetMOCluster(qt.ServiceID()).GetCNService( - clusterservice.NewServiceIDSelector(target), - func(cn metadata.CNService) bool { - req := qt.NewRequest(querypb.CmdMethod_SetProtocolVersion) - req.SetProtocolVersion = &querypb.SetProtocolVersionRequest{ - Version: version, - } - ctx, cancel := context.WithTimeoutCause(context.Background(), time.Second, moerr.CauseTransferToCN) - defer cancel() - - resp, err = qt.SendMessage(ctx, cn.QueryAddress, req) - err = moerr.AttachCause(ctx, err) - return true +func transferToCN( + qt qclient.QueryClient, + target string, + version int64, + activationTargets []string, +) (resp *querypb.Response, err error) { + cluster := clusterservice.GetMOCluster(qt.ServiceID()) + var selected metadata.CNService + if version >= defines.MORPCVersion44 { + refresher, refreshOK := cluster.(clusterservice.AuthoritativeRefresher) + if !refreshOK { + return nil, moerr.NewInternalErrorNoCtx( + "CN cluster service does not support authoritative activation recovery") + } + ctx, cancel := context.WithTimeoutCause( + context.Background(), time.Minute, moerr.CauseTransferToCN) + defer cancel() + if err := refresher.Refresh(ctx); err != nil { + return nil, moerr.AttachCause(ctx, err) + } + if err := clusterservice.GetCNServiceRawWithContext( + ctx, cluster, clusterservice.NewServiceIDSelector(target), func(cn metadata.CNService) bool { + selected = cn + return false + }); err != nil { + return nil, moerr.AttachCause(ctx, err) + } + if selected.ServiceID != target || selected.ViewMetadataAdmissionGeneration == 0 || + selected.QueryAddress == "" { + return nil, moerr.NewInternalErrorNoCtxf( + "no authoritative CN activation target %s with valid generation and query address", target) + } + } else { + cluster.GetCNService(clusterservice.NewServiceIDSelector(target), func(cn metadata.CNService) bool { + selected = cn + return false }) - if err != nil { - return nil, err } - return resp, nil + if selected.QueryAddress == "" { + return nil, nil + } + + req := qt.NewRequest(querypb.CmdMethod_SetProtocolVersion) + req.SetProtocolVersion = &querypb.SetProtocolVersionRequest{ + Version: version, + DDLVisibilityActivationTargets: append([]string(nil), activationTargets...), + DDLVisibilityTargetGeneration: selected.ViewMetadataAdmissionGeneration, + DDLVisibilityTargetQueryAddress: selected.QueryAddress, + } + // Live protocol activation may withdraw CN ingress and wait for a bounded + // logtail frontier fence before acknowledging the transition. + ctx, cancel := context.WithTimeoutCause(context.Background(), time.Minute, moerr.CauseTransferToCN) + defer cancel() + resp, err = qt.SendMessage(ctx, selected.QueryAddress, req) + return resp, moerr.AttachCause(ctx, err) } diff --git a/pkg/sql/plan/function/ctl/cmd_rpc_version_test.go b/pkg/sql/plan/function/ctl/cmd_rpc_version_test.go index 1fd53a045b270..0b8ac3963193c 100644 --- a/pkg/sql/plan/function/ctl/cmd_rpc_version_test.go +++ b/pkg/sql/plan/function/ctl/cmd_rpc_version_test.go @@ -17,6 +17,9 @@ package ctl import ( "context" "fmt" + "strings" + "sync" + "sync/atomic" "testing" "time" @@ -142,16 +145,147 @@ func requireVersionValue(t *testing.T, version int64) { type addressRecordingQueryClient struct { testQClient - address string + address string + deadline time.Time + request *query.Request } func (c *addressRecordingQueryClient) SendMessage( - _ context.Context, address string, _ *query.Request, + ctx context.Context, address string, req *query.Request, ) (*query.Response, error) { c.address = address + c.deadline, _ = ctx.Deadline() + c.request = req return nil, moerr.NewInternalErrorNoCtx("send error") } +type concurrentProtocolQueryClient struct { + testQClient + started atomic.Int32 + both chan struct{} + mu sync.Mutex + requests [][]string + releases atomic.Int32 +} + +func (c *concurrentProtocolQueryClient) SendMessage( + _ context.Context, + _ string, + req *query.Request, +) (*query.Response, error) { + c.mu.Lock() + c.requests = append(c.requests, + append([]string(nil), req.SetProtocolVersion.DDLVisibilityActivationTargets...)) + c.mu.Unlock() + if c.started.Add(1) == 2 { + close(c.both) + } + <-c.both + return &query.Response{SetProtocolVersion: &query.SetProtocolVersionResponse{ + Version: req.SetProtocolVersion.Version, + }}, nil +} + +func (c *concurrentProtocolQueryClient) NewRequest(method query.CmdMethod) *query.Request { + return &query.Request{CmdMethod: method} +} + +func (c *concurrentProtocolQueryClient) Release(*query.Response) { c.releases.Add(1) } + +func TestHandleSetProtocolVersionDispatchesCompleteTargetSetConcurrently(t *testing.T) { + targets := []string{"activation-cn-1", "activation-cn-2"} + rt := runtime.DefaultRuntime() + runtime.SetupServiceBasedRuntime("", rt) + mc := clusterservice.NewMOCluster( + "", + nil, + 3*time.Second, + clusterservice.WithDisableRefresh(), + clusterservice.WithServices( + []metadata.CNService{ + {ServiceID: targets[0], QueryAddress: "cn-1:6001", WorkState: metadata.WorkState_Working, + ViewMetadataAdmissionGeneration: 1, DDLVisibilityBarrierReady: true}, + {ServiceID: targets[1], QueryAddress: "cn-2:6001", WorkState: metadata.WorkState_Working, + ViewMetadataAdmissionGeneration: 2, DDLVisibilityBarrierReady: true}, + }, + nil, + ), + ) + defer mc.Close() + rt.SetGlobalVariables(runtime.ClusterService, mc) + qcli := &concurrentProtocolQueryClient{both: make(chan struct{})} + proc := &process.Process{Base: &process.BaseProcess{QueryClient: qcli}} + + result, err := handleSetProtocolVersion( + proc, cn, strings.Join(targets, ",")+":44", nil) + require.NoError(t, err) + require.Equal(t, "activation-cn-1:44, activation-cn-2:44", result.Data) + require.Equal(t, int32(2), qcli.started.Load()) + require.Equal(t, int32(2), qcli.releases.Load()) + qcli.mu.Lock() + defer qcli.mu.Unlock() + require.Len(t, qcli.requests, 2) + for _, requestTargets := range qcli.requests { + require.Equal(t, targets, requestTargets) + } + + _, err = handleSetProtocolVersion(proc, cn, "activation-cn-1,activation-cn-1:44", nil) + require.ErrorContains(t, err, "duplicated") +} + +func TestHandleSetProtocolVersionRejectsOmittedAuthoritativeCN(t *testing.T) { + rt := runtime.DefaultRuntime() + runtime.SetupServiceBasedRuntime("", rt) + mc := clusterservice.NewMOCluster( + "", nil, 3*time.Second, clusterservice.WithDisableRefresh(), + clusterservice.WithServices([]metadata.CNService{ + {ServiceID: "new-cn", QueryAddress: "new:6001", ViewMetadataAdmissionGeneration: 1, + DDLVisibilityBarrierReady: true}, + {ServiceID: "legacy-public-cn", QueryAddress: "legacy:6001", ViewMetadataAdmissionGeneration: 2, + ViewMetadataIngressReady: true}, + }, nil), + ) + defer mc.Close() + rt.SetGlobalVariables(runtime.ClusterService, mc) + proc := &process.Process{Base: &process.BaseProcess{QueryClient: &concurrentProtocolQueryClient{both: make(chan struct{})}}} + + _, err := handleSetProtocolVersion(proc, cn, "new-cn:44", nil) + require.ErrorContains(t, err, "does not match authoritative CN membership") +} + +func TestTransferToCNAllowsActivationFence(t *testing.T) { + const serviceID = "activation-cn" + rt := runtime.DefaultRuntime() + runtime.SetupServiceBasedRuntime("", rt) + mc := clusterservice.NewMOCluster( + "", + nil, + 3*time.Second, + clusterservice.WithDisableRefresh(), + clusterservice.WithServices( + []metadata.CNService{{ + ServiceID: serviceID, QueryAddress: "activation-cn:6001", + WorkState: metadata.WorkState_Draining, ViewMetadataAdmissionGeneration: 1, + }}, + nil, + ), + ) + defer mc.Close() + rt.SetGlobalVariables(runtime.ClusterService, mc) + + qcli := &addressRecordingQueryClient{} + started := time.Now() + _, err := transferToCN(qcli, serviceID, defines.MORPCVersion44, []string{serviceID}) + require.Error(t, err) + require.Equal(t, "activation-cn:6001", qcli.address) + require.Equal(t, []string{serviceID}, + qcli.request.SetProtocolVersion.DDLVisibilityActivationTargets) + require.Equal(t, uint64(1), qcli.request.SetProtocolVersion.DDLVisibilityTargetGeneration) + require.Equal(t, "activation-cn:6001", + qcli.request.SetProtocolVersion.DDLVisibilityTargetQueryAddress) + require.WithinDuration(t, started.Add(time.Minute), qcli.deadline, time.Second) +} + func Test_transferToTN(t *testing.T) { rt := runtime.DefaultRuntime() diff --git a/pkg/tests/issues/issue_277xx_ddl_consistency_test.go b/pkg/tests/issues/issue_277xx_ddl_consistency_test.go index 4cbd9eca72679..27ea07a16ae40 100644 --- a/pkg/tests/issues/issue_277xx_ddl_consistency_test.go +++ b/pkg/tests/issues/issue_277xx_ddl_consistency_test.go @@ -24,7 +24,9 @@ import ( "time" mysql "github.com/go-sql-driver/mysql" + "github.com/matrixorigin/matrixone/pkg/cnservice" "github.com/matrixorigin/matrixone/pkg/embed" + "github.com/matrixorigin/matrixone/pkg/util/fault" "github.com/stretchr/testify/require" ) @@ -42,6 +44,47 @@ func TestIssue277xxDDLConsistency(t *testing.T) { db1 := openIssue277xxDB(t, cn1.GetServiceConfig().CN.Frontend.Port) defer db1.Close() + t.Run("27743 automatic cross-CN DDL visibility", func(t *testing.T) { + const database = "issue_27743_automatic_ddl_visibility" + targets := cn0.ServiceID() + "," + cn1.ServiceID() + var activation string + require.NoError(t, db0.QueryRowContext(ctx, + "select mo_ctl('cn', 'SetProtocolVersion', ?)", targets+":44").Scan(&activation)) + require.Contains(t, activation, cn0.ServiceID()+":44") + require.Contains(t, activation, cn1.ServiceID()+":44") + // SetProtocolVersion returns success only after both target RPCs report + // v44 and HAKeeper acknowledges the atomically committed v44 epoch. + + resetIssue277xxDatabase(t, ctx, db0, database) + defer execSQLMaybe(t, ctx, db0, "drop database if exists `"+database+"`") + + // Prove this public path actually executes SyncCommitV2: an injected + // receiver failure must make CREATE fail after v44 activation. + require.True(t, fault.Enable()) + defer fault.Disable() + faultPointRemoved := false + defer func() { + if !faultPointRemoved { + _, _ = fault.RemoveFaultPoint(context.Background(), cnservice.DDLVisibilitySyncCommitFaultPoint) + } + }() + require.NoError(t, fault.AddFaultPoint(ctx, + cnservice.DDLVisibilitySyncCommitFaultPoint, "1:1::", "return", 0, "expected", false)) + _, err := db0.ExecContext(ctx, + "create table `"+database+"`.`must_fail_sync` (id int primary key)") + require.ErrorContains(t, err, "injected DDL visibility sync commit error") + _, removeErr := fault.RemoveFaultPoint(ctx, cnservice.DDLVisibilitySyncCommitFaultPoint) + require.NoError(t, removeErr) + faultPointRemoved = true + + // Do not issue SYNCCOMMIT or retry the read. The v44 DDL commit contract + // must make the first fresh CN1 snapshot observe CN0's CREATE TABLE. + execSQLRequire(t, ctx, db0, + "create table `"+database+"`.`t` (id int primary key, payload varchar(32))") + require.Equal(t, 0, queryIssue277xxInt(t, ctx, db1, + "select count(*) from `"+database+"`.`t`")) + }) + t.Run("27735 qualified alter drop index without default database", func(t *testing.T) { const database = "issue_27735_qualified_drop" resetIssue277xxDatabase(t, ctx, db0, database) diff --git a/pkg/txn/client/client.go b/pkg/txn/client/client.go index 5431bd3abe3d3..d5b6bde7473bc 100644 --- a/pkg/txn/client/client.go +++ b/pkg/txn/client/client.go @@ -962,7 +962,7 @@ func (client *txnClient) GetLatestCommitTS() timestamp.Timestamp { func (client *txnClient) SyncLatestCommitTS(ts timestamp.Timestamp) { client.updateLastCommitTS(context.TODO(), nil, TxnEvent{Txn: txn.TxnMeta{CommitTS: ts}}, nil) - if client.timestampWaiter != nil { + if client.timestampWaiter != nil && client.timestampWaiter.LatestTS().Less(ts) { ctx, cancel := context.WithTimeoutCause(context.Background(), time.Minute*5, moerr.CauseSyncLatestCommitT) defer cancel() for { diff --git a/pkg/txn/client/client_test.go b/pkg/txn/client/client_test.go index 38f3cbd01e8d1..d5a49cb984262 100644 --- a/pkg/txn/client/client_test.go +++ b/pkg/txn/client/client_test.go @@ -212,6 +212,11 @@ type snapshotTimestampWaiter struct { waitForContext bool } +type appliedTimestampWaiter struct { + latest timestamp.Timestamp + getCalls int +} + func (w *snapshotTimestampWaiter) GetTimestamp( ctx context.Context, ts timestamp.Timestamp, @@ -229,6 +234,18 @@ func (*snapshotTimestampWaiter) NotifyLatestCommitTS(timestamp.Timestamp) {} func (*snapshotTimestampWaiter) Close() {} func (*snapshotTimestampWaiter) LatestTS() timestamp.Timestamp { return timestamp.Timestamp{} } +func (w *appliedTimestampWaiter) GetTimestamp( + context.Context, + timestamp.Timestamp, +) (timestamp.Timestamp, error) { + w.getCalls++ + return w.latest.Next(), nil +} + +func (*appliedTimestampWaiter) NotifyLatestCommitTS(timestamp.Timestamp) {} +func (*appliedTimestampWaiter) Close() {} +func (w *appliedTimestampWaiter) LatestTS() timestamp.Timestamp { return w.latest } + // selectedSuccessTimestampWaiter models the valid notify-vs-cancel ordering // where notification wins the waiter's select, but the successful return is // delayed until after the transaction client has closed. @@ -273,6 +290,42 @@ func TestAdjustClient(t *testing.T) { } } +func TestSyncLatestCommitTS(t *testing.T) { + targetTS := timestamp.Timestamp{PhysicalTime: 100, LogicalTime: 7} + + t.Run("without timestamp waiter", func(t *testing.T) { + client := &txnClient{} + + client.SyncLatestCommitTS(targetTS) + + require.Equal(t, targetTS, client.GetLatestCommitTS()) + require.Equal(t, uint64(1), client.GetSyncLatestCommitTSTimes()) + }) + + t.Run("skips an already applied timestamp waiter", func(t *testing.T) { + waiter := &appliedTimestampWaiter{latest: targetTS} + client := &txnClient{timestampWaiter: waiter} + + client.SyncLatestCommitTS(targetTS) + + require.Zero(t, waiter.getCalls) + require.Equal(t, targetTS, client.GetLatestCommitTS()) + require.Equal(t, uint64(1), client.GetSyncLatestCommitTSTimes()) + }) + + t.Run("waits for an unapplied timestamp", func(t *testing.T) { + waiter := &snapshotTimestampWaiter{} + client := &txnClient{timestampWaiter: waiter} + + client.SyncLatestCommitTS(targetTS) + + require.Equal(t, 1, waiter.called) + require.Equal(t, targetTS, waiter.got) + require.Equal(t, targetTS, client.GetLatestCommitTS()) + require.Equal(t, uint64(1), client.GetSyncLatestCommitTSTimes()) + }) +} + func TestNewWithSnapshotWaitsForLocalLogtail(t *testing.T) { rt := runtime.NewRuntime(metadata.ServiceType_CN, "", logutil.GetPanicLogger(), diff --git a/proto/logservice.proto b/proto/logservice.proto index 8f7a546f641bc..335b3df30e0c0 100644 --- a/proto/logservice.proto +++ b/proto/logservice.proto @@ -22,6 +22,7 @@ option go_package = "github.com/matrixorigin/matrixone/pkg/pb/logservice"; import "github.com/gogo/protobuf/gogoproto/gogo.proto"; import "google/protobuf/timestamp.proto"; import "metadata.proto"; +import "timestamp.proto"; option (gogoproto.goproto_enum_prefix_all) = false; option (gogoproto.sizer_all) = false; @@ -61,6 +62,15 @@ message CNStore { uint64 ViewMetadataAdmissionGeneration = 18; uint64 ViewMetadataObservedEpoch = 19; bool ViewMetadataAdmissionReady = 20; + // DDLVisibilityBarrierReady means QueryService can accept the versioned DDL + // visibility fence even when public SQL ingress is not admitted yet. + bool DDLVisibilityBarrierReady = 21; + // ViewMetadataIngressReady is retained for raw control-plane inventory. + bool ViewMetadataIngressReady = 22; + int64 DDLVisibilityDeployedProtocol = 23; + bool DDLVisibilityActivationPrepared = 24; + bool DDLVisibilityActivationFenced = 25; + timestamp.Timestamp DDLVisibilityFrontier = 26 [(gogoproto.nullable) = false]; } message TNStore { @@ -92,6 +102,9 @@ message LogStore { repeated LogReplicaInfo Replicas = 5 [(gogoproto.nullable) = false]; ConfigData ConfigData = 6; Locality Locality = 7 [(gogoproto.nullable) = false]; + // DDLVisibilityDeployedProtocolSupported proves the hosted HAKeeper RSM can + // persist and return the cluster-wide DDL deployment epoch. + bool DDLVisibilityDeployedProtocolSupported = 8; } // LogShardInfo contains information a log shard. @@ -139,6 +152,12 @@ message Resource { } // CNStoreHeartbeat is the periodic message sent tp the HAKeeper by CN stores. +message DDLVisibilityActivationTarget { + string ServiceID = 1; + uint64 Generation = 2; + string QueryAddress = 3; +} + message CNStoreHeartbeat { string UUID = 1; string ServiceAddress = 2; @@ -170,6 +189,17 @@ message CNStoreHeartbeat { // ViewMetadataIngressReady is published only after every public CN ingress // listener has started successfully. Admission alone is not routability. bool ViewMetadataIngressReady = 24; + bool DDLVisibilityBarrierReady = 25; + // DDLVisibilityDeployedProtocol reports only a locally committed deployment + // marker. Provisional activation must never advance the cluster-wide epoch. + int64 DDLVisibilityDeployedProtocol = 26; + // Non-empty only for the transition that atomically validates the complete + // CN membership tuple set and commits DDLVisibilityDeployedProtocol. + repeated DDLVisibilityActivationTarget DDLVisibilityEpochCommitTargets = 27 + [(gogoproto.nullable) = false]; + bool DDLVisibilityActivationPrepared = 28; + bool DDLVisibilityActivationFenced = 29; + timestamp.Timestamp DDLVisibilityFrontier = 30 [(gogoproto.nullable) = false]; } // CNAllocateID is the periodic message sent tp the HAKeeper by CN stores. @@ -211,6 +241,9 @@ message LogStoreHeartbeat { // this Log store. It remains stable across process restarts and changes when // the store starts from a new data directory. string StoreIncarnation = 11; + // This gates the HAKeeper RSM schema used by the durable DDL deployment + // epoch across both voting and non-voting replicas. + bool DDLVisibilityDeployedProtocolSupported = 12; }; // TNShardInfo contains information of a launched TN shard. @@ -639,6 +672,12 @@ message CommandBatch { // ViewMetadataAdmission is the authoritative snapshot for the heartbeat // sender. Old services ignore this additive field. ViewMetadataAdmission ViewMetadataAdmission = 5; + // DDLVisibilityDeployedProtocol is returned from the same replicated + // heartbeat transition that conditionally publishes CN ingress. + int64 DDLVisibilityDeployedProtocol = 6; + // DDLVisibilityFrontier is the durable cluster maximum after applying this + // heartbeat transition. New CNs require it as publication acknowledgement. + timestamp.Timestamp DDLVisibilityFrontier = 7; } // ViewMetadataAdmission is the durable cluster admission snapshot. Generation @@ -697,12 +736,24 @@ message CNStoreInfo { uint64 ViewMetadataRevalidatedEpoch = 24; bool ViewMetadataAdmissionReady = 25; bool ViewMetadataIngressReady = 26; + bool DDLVisibilityBarrierReady = 27; + int64 DDLVisibilityDeployedProtocol = 28; + bool DDLVisibilityActivationPrepared = 29; + bool DDLVisibilityActivationFenced = 30; + timestamp.Timestamp DDLVisibilityFrontier = 31 [(gogoproto.nullable) = false]; } // CNState contains all CN details known to the HAKeeper. message CNState { - // Stores is keyed by CN store UUID. + // Stores is keyed by CN store UUID. map Stores = 1 [(gogoproto.nullable) = false]; + // DDLVisibilityDeployedProtocol is a monotonic cluster deployment epoch. + // Once a complete-target cut commits, markerless replacement/scale-out CNs + // must join that fence instead of reopening on an older protocol. + int64 DDLVisibilityDeployedProtocol = 2; + // DDLVisibilityFrontier is the monotonic cluster-lifetime maximum committed + // DDL timestamp. It survives CN restart, replacement, and store removal. + timestamp.Timestamp DDLVisibilityFrontier = 3 [(gogoproto.nullable) = false]; } @@ -768,6 +819,8 @@ message ClusterDetails { repeated ProxyStore ProxyStores = 4 [(gogoproto.nullable) = false]; repeated DeletedStore DeletedStores = 5 [(gogoproto.nullable) = false]; ViewMetadataAdmission ViewMetadataAdmission = 6; + int64 DDLVisibilityDeployedProtocol = 7; + timestamp.Timestamp DDLVisibilityFrontier = 8 [(gogoproto.nullable) = false]; } // ClusterInfo provides a global view of all shards in the cluster. It @@ -808,6 +861,7 @@ message LogStoreInfo { bool CommandDeliverySupported = 9; bool ViewMetadataAdmissionSupported = 10; string StoreIncarnation = 11; + bool DDLVisibilityDeployedProtocolSupported = 12; } message LogState { diff --git a/proto/metadata.proto b/proto/metadata.proto index 0930ad8d75800..0dffcc56c0621 100644 --- a/proto/metadata.proto +++ b/proto/metadata.proto @@ -113,6 +113,9 @@ message CNStore { string UUID = 1; // Role CN role CNRole Role = 2; + // DDLVisibilityDeployedProtocol is the durable activation level restored + // before this CN admits SQL after an ordinary restart. + int64 DDLVisibilityDeployedProtocol = 3; } // CNService cn service metadata @@ -142,6 +145,12 @@ message CNService { uint64 ViewMetadataAdmissionGeneration = 13; bool ViewMetadataAdmissionReady = 14; uint64 ViewMetadataObservedEpoch = 15; + // DDLVisibilityBarrierReady is published after QueryService is listening. + // A CN is not publicly routable until its startup DDL frontier fence completes. + bool DDLVisibilityBarrierReady = 16; + // ViewMetadataIngressReady is retained in raw control-plane inventory so + // protocol activation can verify withdrawal/publication without routing it. + bool ViewMetadataIngressReady = 17; } // TNService tn service metadata diff --git a/proto/query.proto b/proto/query.proto index 587c1223493b9..a0c337fe99a22 100644 --- a/proto/query.proto +++ b/proto/query.proto @@ -108,6 +108,9 @@ enum CmdMethod { IcebergCacheInvalidate = 38; // MongoDBClientRetire drains obsolete MongoDB driver clients after catalog DDL. MongoDBClientRetire = 39; + // SyncCommitV2 waits for local logtail application with the request context. + // Unlike legacy SyncCommit, cancellation is returned to the sender. + SyncCommitV2 = 40; } // QueryRequest is the common query request. It contains the query @@ -161,11 +164,23 @@ message GetProtocolVersionRequest { message GetProtocolVersionResponse { // ProtocolVersion is the version of the protocol int64 Version = 1; + // DDLVisibilityActivationPrepared means this CN has blocked and drained DDL + // producers before exposing the v36 sender capability. + bool DDLVisibilityActivationPrepared = 2; + // DDLVisibilityActivationFenced means this CN applied the converged peer + // frontier while every participant's DDL producers were blocked. + bool DDLVisibilityActivationFenced = 3; } message SetProtocolVersionRequest { // ProtocolVersion is the version of the protocol int64 Version = 1; + // DDLVisibilityActivationTargets is the complete CN set participating in a + // coordinated live activation. It is empty for ordinary version updates. + repeated string DDLVisibilityActivationTargets = 2; + // The authoritative raw-inventory identity selected by the control plane. + uint64 DDLVisibilityTargetGeneration = 3; + string DDLVisibilityTargetQueryAddress = 4; } message SetProtocolVersionResponse {