Skip to content

Commit 68e6e81

Browse files
committed
feat(api): add EtcdDefrag API type and docs
Introduces a dedicated EtcdDefrag resource for operator-driven backend defragmentation, instead of the spec.defrag-in-EtcdClusterSpec shape rejected in #361's review. One-shot and run-to-completion, modeled on EtcdSnapshot: spec.clusterRef, an optional rule (with a reclaimable floor so a full-but-unfragmented backend isn't defragmented for nothing), and a status carrying per-member outcomes. Recurring runs are driven externally, as with EtcdSnapshot. This lands the API type, generated deepcopy/CRD, and user docs so the reconciling controller can follow as a self-contained change. Until that lands the resource is inert (documented as such). No changes to EtcdClusterSpec. Refs #221, #357; supersedes the #361 approach. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com>
1 parent c05e754 commit 68e6e81

5 files changed

Lines changed: 758 additions & 0 deletions

File tree

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ For step-by-step setup, RBAC, image versions, and teardown see [docs/installatio
7676
- **[Installation](docs/installation.md)** — deploy the operator, create your first cluster, networking pitfalls, upgrades.
7777
- **[Concepts](docs/concepts.md)** — design rationale: locking pattern, single-seed bootstrap, GenerateName naming, scale-to-zero mechanics, conditions reference.
7878
- **[Operations](docs/operations.md)** — runbook for day-2: scaling, pausing/resuming, decoding conditions, escalating stuck reconciles, broken-member recovery.
79+
- **[Defragmentation](docs/etcd-defrag.md)** — the `EtcdDefrag` resource: reclaiming etcd backend disk, one-shot and scheduled, and its safety model.
7980
- **[Migration](docs/migration.md)** — moving onto this operator from the legacy aenix operator; tracks behavioural changes that need an explicit migration step — currently the BYO root-credentials requirement when enabling auth.
8081

8182
## Testing

api/v1alpha2/etcddefrag_types.go

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
/*
2+
Copyright 2023 Timofey Larkin.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package v1alpha2
18+
19+
import (
20+
corev1 "k8s.io/api/core/v1"
21+
"k8s.io/apimachinery/pkg/api/resource"
22+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
23+
)
24+
25+
// EtcdDefragSpec is the desired state of an EtcdDefrag: a one-shot request to
26+
// defragment an EtcdCluster's members.
27+
type EtcdDefragSpec struct {
28+
// ClusterRef names the EtcdCluster (same namespace) to defragment.
29+
ClusterRef corev1.LocalObjectReference `json:"clusterRef"`
30+
31+
// Rule optionally skips members not worth defragmenting, so a periodic
32+
// EtcdDefrag is cheap when nothing is fragmented. Absent means every member
33+
// is defragmented unconditionally (an explicit "do it now").
34+
// +optional
35+
Rule *DefragRule `json:"rule,omitempty"`
36+
37+
// TTLSecondsAfterFinished garbage-collects this record the given number of
38+
// seconds after it reaches a terminal phase — for objects an external
39+
// scheduler stamps out. Absent means the record is kept as history.
40+
// +kubebuilder:validation:Minimum=0
41+
// +optional
42+
TTLSecondsAfterFinished *int32 `json:"ttlSecondsAfterFinished,omitempty"`
43+
}
44+
45+
// DefragRule decides whether a member is worth defragmenting. A defrag can only
46+
// reclaim DbSize-DbSizeInUse, so that reclaimable amount is the always-applied
47+
// floor — what stops a full-but-unfragmented backend (DbSize ≈ DbSizeInUse near
48+
// the quota) from being defragmented forever for nothing.
49+
//
50+
// +kubebuilder:validation:XValidation:rule="!has(self.freeSpaceAbove) || quantity(self.freeSpaceAbove).isGreaterThan(quantity('0'))",message="freeSpaceAbove must be greater than 0"
51+
// +kubebuilder:validation:XValidation:rule="!has(self.minReclaim) || quantity(self.minReclaim).isGreaterThan(quantity('0'))",message="minReclaim must be greater than 0"
52+
type DefragRule struct {
53+
// FreeSpaceAbove defragments a member whose reclaimable space
54+
// (DbSize-DbSizeInUse) exceeds this. The primary, always-applied gate.
55+
// Absent means the built-in default (200Mi).
56+
// +optional
57+
FreeSpaceAbove *resource.Quantity `json:"freeSpaceAbove,omitempty"`
58+
59+
// QuotaUsageAbove: when DbSize exceeds this fraction of the backend quota
60+
// (approaching NOSPACE), lower the reclaimable floor to MinReclaim so small
61+
// wins are taken under pressure. A member is never defragmented when its
62+
// reclaimable space is below MinReclaim. Integer percent 1..100 with a "%"
63+
// suffix, e.g. "80%".
64+
// +kubebuilder:validation:Pattern=`^([1-9][0-9]?|100)%$`
65+
// +optional
66+
QuotaUsageAbove string `json:"quotaUsageAbove,omitempty"`
67+
68+
// MinReclaim floors the quota arm: even under quota pressure, skip a member
69+
// that would reclaim less than this. Absent means the built-in default
70+
// (32Mi).
71+
// +optional
72+
MinReclaim *resource.Quantity `json:"minReclaim,omitempty"`
73+
}
74+
75+
// EtcdDefragPhase is the lifecycle phase of an EtcdDefrag.
76+
type EtcdDefragPhase string
77+
78+
const (
79+
// EtcdDefragPhasePending is the initial phase: the request is queued.
80+
// Defragmentations are serialized per cluster, so a request waits here while
81+
// another is running against the same EtcdCluster.
82+
EtcdDefragPhasePending EtcdDefragPhase = "Pending"
83+
// EtcdDefragPhaseRunning means the member sweep is in progress.
84+
EtcdDefragPhaseRunning EtcdDefragPhase = "Running"
85+
// EtcdDefragPhaseDeferred means a defragmentation is due but the cluster is
86+
// not fully healthy, so it is being retried rather than forced (defrag must
87+
// never risk quorum).
88+
EtcdDefragPhaseDeferred EtcdDefragPhase = "Deferred"
89+
// EtcdDefragPhaseComplete means the sweep finished; see status.members for
90+
// per-member outcomes.
91+
EtcdDefragPhaseComplete EtcdDefragPhase = "Complete"
92+
// EtcdDefragPhaseFailed means the sweep could not complete.
93+
EtcdDefragPhaseFailed EtcdDefragPhase = "Failed"
94+
)
95+
96+
// EtcdDefragStatus is the observed state of an EtcdDefrag.
97+
type EtcdDefragStatus struct {
98+
// Phase is the high-level lifecycle phase.
99+
// +optional
100+
Phase EtcdDefragPhase `json:"phase,omitempty"`
101+
102+
// StartedAt is when the sweep began.
103+
// +optional
104+
StartedAt *metav1.Time `json:"startedAt,omitempty"`
105+
106+
// CompletedAt is when the sweep reached a terminal phase.
107+
// +optional
108+
CompletedAt *metav1.Time `json:"completedAt,omitempty"`
109+
110+
// Defragmented counts members actually defragmented this run.
111+
// +optional
112+
Defragmented int32 `json:"defragmented,omitempty"`
113+
114+
// Members holds the per-member outcome of the sweep, in processing order
115+
// (followers before the leader).
116+
// +optional
117+
// +listType=map
118+
// +listMapKey=name
119+
Members []MemberDefragStatus `json:"members,omitempty"`
120+
121+
// Conditions represent the latest available observations.
122+
// +optional
123+
Conditions []metav1.Condition `json:"conditions,omitempty"`
124+
}
125+
126+
// MemberDefragStatus is the outcome of defragmenting a single member.
127+
type MemberDefragStatus struct {
128+
// Name is the EtcdMember this row describes.
129+
Name string `json:"name"`
130+
131+
// Role at the time it was processed: "leader" or "follower".
132+
// +optional
133+
Role string `json:"role,omitempty"`
134+
135+
// Outcome is one of Pending, Skipped, Defragmented, Failed.
136+
// +optional
137+
Outcome string `json:"outcome,omitempty"`
138+
139+
// Reason qualifies the outcome (e.g. BelowThreshold, ClusterNotHealthy,
140+
// RPCError).
141+
// +optional
142+
Reason string `json:"reason,omitempty"`
143+
144+
// DBSizeBefore is the member's physical backend size before defragmenting.
145+
// +optional
146+
DBSizeBefore int64 `json:"dbSizeBefore,omitempty"`
147+
148+
// DBSizeAfter is the physical backend size after defragmenting.
149+
// +optional
150+
DBSizeAfter int64 `json:"dbSizeAfter,omitempty"`
151+
152+
// ReclaimedBytes is DBSizeBefore-DBSizeAfter for a completed defrag.
153+
// +optional
154+
ReclaimedBytes int64 `json:"reclaimedBytes,omitempty"`
155+
156+
// FinishedAt is when this member was processed.
157+
// +optional
158+
FinishedAt *metav1.Time `json:"finishedAt,omitempty"`
159+
}
160+
161+
// +kubebuilder:object:root=true
162+
// +kubebuilder:subresource:status
163+
// +kubebuilder:printcolumn:name="Cluster",type=string,JSONPath=`.spec.clusterRef.name`
164+
// +kubebuilder:printcolumn:name="Phase",type=string,JSONPath=`.status.phase`
165+
// +kubebuilder:printcolumn:name="Defragmented",type=integer,JSONPath=`.status.defragmented`
166+
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
167+
168+
// EtcdDefrag is the Schema for the etcddefrags API. It requests a one-shot,
169+
// run-to-completion defragmentation of an EtcdCluster's members. Like
170+
// EtcdSnapshot it is a record: the operator drives it through status.phase and
171+
// it never re-runs. Recurring defragmentation is driven from outside by
172+
// creating EtcdDefrag objects on a schedule.
173+
type EtcdDefrag struct {
174+
metav1.TypeMeta `json:",inline"`
175+
metav1.ObjectMeta `json:"metadata,omitempty"`
176+
177+
Spec EtcdDefragSpec `json:"spec,omitempty"`
178+
Status EtcdDefragStatus `json:"status,omitempty"`
179+
}
180+
181+
// +kubebuilder:object:root=true
182+
183+
// EtcdDefragList contains a list of EtcdDefrag.
184+
type EtcdDefragList struct {
185+
metav1.TypeMeta `json:",inline"`
186+
metav1.ListMeta `json:"metadata,omitempty"`
187+
Items []EtcdDefrag `json:"items"`
188+
}
189+
190+
func init() {
191+
SchemeBuilder.Register(&EtcdDefrag{}, &EtcdDefragList{})
192+
}

api/v1alpha2/zz_generated.deepcopy.go

Lines changed: 166 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)