Skip to content
opendefensecloudPublic

About

Development tools for opendefense.cloud projects

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

dev-kit

A library that provides a pre-configured development environment.

Copy the files from example/ into your project and adjust them for your needs.

Features

  • Nix flake with a development shell (Go, pre-commit hooks, cosign, trivy)
  • direnv integration for automatic environment activation
  • Make targets for common development tasks
  • tools.lock for pinning Go binaries to specific versions

Usage

Make targets

The included common.mk provides:

Target Description
help Display all available targets
clean Remove the bin/ directory
fmt Format every file in the repository
lint Check formatting, then run shellcheck and, when the repository has a go.mod, golangci-lint
fmt-all Formatting only (what fmt calls)
lint-all Fail if any file is not formatted (what lint calls)
mod Run go mod tidy, download, and verify
golangci-lint Run golangci-lint
shellcheck Run shellcheck on shell scripts
scan Scan for vulnerabilities using osv-scanner
setup-local-cluster Create a Kind cluster for local development
repo-settings Reconcile GitHub repository settings
envtest-binaries-sideload Populate the envtest cache from upstream K8s/etcd releases
check-go-version Assert every Go version pin in the repository agrees
check-dev-kit-pins Assert every dev-kit reference names the same release

Formatting and linting

Formatting is opt-in. Set it in the project Makefile before -include common.mk, next to DEV_KIT_VERSION:

DEV_KIT_FORMATTING := on

While it is off, no formatting runs: fmt does nothing beyond whatever the project's own fmt: rule does, and lint runs shellcheck and golangci-lint exactly as before, fetching only .golangci.yml.

This is opt-in because a repository that still defines its own fmt: or lint: recipe would otherwise have fmt-all/lint-all merged in as prerequisites silently on its next DEV_KIT_VERSION bump — make appends prerequisites without warning — and start failing on every YAML, Markdown and shell file it has never formatted. Run the one-time make fmt-all commit first, then switch it on deliberately.

One formatter front-end for the whole repository. treefmt drives gofmt (via golangci-lint fmt, so import order stays configured in one place), yamlfmt, shfmt and mdformat.

make fmt     # writes
make lint    # fails on drift, then shellcheck and golangci-lint (Go repos)

The Go steps are conditional on a go.mod in the repository root: without one, lint skips golangci-lint, fmt does not install it, .golangci.yml is not fetched, and treefmt is passed --allow-missing-formatter so it does not insist on a formatter it will never call. None of that needs configuring by hand.

treefmt.toml, .golangci.yml and .editorconfig are fetched from dev-kit at the pinned DEV_KIT_VERSION the first time they are needed, and refetched when that version changes. A repository that commits its own copy of one of these files keeps it. The fetch rule only fires for a file that is absent, and an alternate spelling counts as an override (.golangci.yaml, .treefmt.toml). Add the bootstrapped ones to .gitignore, together with .common.mk-configs, next to the existing common.mk entry — .common.mk-configs records what was fetched so that a DEV_KIT_VERSION bump only ever removes files this mechanism created, never a hand-written override that is not committed yet.

Refresh happens when the DEV_KIT_VERSION string changes. With a moving ref (main) the configs are therefore fetched once and not refreshed again, while common.mk itself keeps updating via its hourly content check.

shfmt reads its style from .editorconfig, so an editor and make fmt cannot disagree about shell scripts.

Repositories extend the entry points by adding prerequisites rather than redefining them, which keeps the recipe in common.mk:

fmt: license-headers
lint: license-headers-check

Variables

Variable Default Description
BUILD_PATH $(shell pwd) Base directory for local binaries
LOCALBIN $(BUILD_PATH)/bin Directory for installed binaries
OSV_SCANNER_CONFIG ./.osv-scanner.toml Path to osv-scanner configuration
DEV_KIT_FORMATTING off on enables the shared fmt/lint formatting. Set it before -include common.mk
DEV_KIT_CONFIGS .editorconfig .golangci.yml treefmt.toml Shared configs to bootstrap; set to empty to opt out. Each file is skipped automatically when the repo overrides or does not need it
TREEFMT_ARGS derived Extra treefmt flags. Defaults to --allow-missing-formatter in repos without a go.mod, empty otherwise
OS $(shell $(GO) env GOOS) Current Operating System
ARCH $(shell $(GO) env GOARCH) Current CPU architecture

Any binary defined in your tools.lock is also available as a Make target (e.g. make $(CONTROLLER_GEN)). Take a look at the variables defined in common.mk for a list of pre-defined binary paths.

To include common.mk into your own Makefile use this snippet or copy the provided Makefile in example/:

DEV_KIT_VERSION := v1.0.0
-include common.mk
common.mk:
  @[ -f .common.mk-download ] || \
		curl --fail -sSL https://raw.githubusercontent.com/opendefensecloud/dev-kit/$(DEV_KIT_VERSION)/common.mk \
		  -o .common.mk-download
	mv .common.mk-download $@
	printf '%s' '$(DEV_KIT_VERSION)' > .common.mk-version
	touch .common.mk-checked

Add your own targets in your project's Makefile like normal:

my-target:
	@echo Hello World

You can override targets provided by common.mk:

scan:
	@my-custom-scanner ./...
$ make scan
Makefile:18: warning: overriding recipe for target 'scan'
common.mk:84: warning: ignoring old recipe for target 'scan'
  scanning...

Go binaries

Define binaries in tools.lock (one per line):

cobra-cli github.com/spf13/cobra-cli@v1.3.0

Install and use them in your Makefile:

COBRA := $(LOCALGOBIN)/cobra-cli
cobra: $(COBRA)
	$(COBRA) help

envtest sideloading

envtest-binaries-sideload populates setup-envtest's cache directly from dl.k8s.io and the etcd GitHub releases, for Kubernetes versions that controller-tools has not yet packaged into its envtest-releases index. Without it, bumping to a fresh Kubernetes release breaks make test until controller-tools catches up. It is a no-op when the version is already cached, so it is safe as a prerequisite of your test target.

Set ENVTEST_K8S_VERSION in your own Makefile; common.mk does not default it. Linux downloads prebuilt binaries; macOS builds kube-apiserver from source when controller-tools has no archive for the requested version, which also needs go on PATH — supplied by goVersion, not by the default dev shell.

Go version consistency

A Go project pins its version in up to three places, and a bump that updates only some of them fails in a workflow that does not mention the file that is wrong:

File Pin What it drives
go.mod go <version> the language version
flake.nix goVersion = "<version>"; the dev shell and CI
Dockerfile FROM ... golang:<tag> the build image

check-go-version asserts they agree, skipping any file the repository does not have — a library without a Dockerfile still passes. Every tracked Dockerfile is read, not just the one at the root, so adding one does not mean remembering to register it. Override GO_MOD, FLAKE_NIX or DOCKERFILE (a space-separated list) for a non-default layout.

A file that exists but whose pin cannot be parsed is an error, never a skip — silently ignoring it is the same drift the check exists to catch.

The Docker tag must be patch-level (1.27.0, not 1.27). A minor-level tag never compares equal to the other two, and Renovate cannot fold it into the same update group, so the image silently stays behind while everything else moves.

Related: the dev shell resolves goVersion from go-overlay, which flake.lock pins. A Go patch released after that lock was written does not exist in the shell yet; the shell says so and names the fix (nix flake update go-overlay) rather than failing with a missing attribute.

Keeping the dev-kit pins in step

A consumer pins dev-kit in up to four places, and only the workflow pins are both immutable and reviewed:

Where Pin Mutable?
.github/workflows/ @<sha> # <tag> no
Makefile DEV_KIT_VERSION yes — a tag
flake.nix the flake input yes — a tag
flake.lock the resolved rev no, but rewritten by nix

check-dev-kit-pins asserts they all name the same release. Add it to your lint target; absent sources are skipped, so a consumer without a flake still passes. Override WORKFLOWS, DEV_KIT_MAKEFILE, FLAKE_NIX or FLAKE_LOCK for a non-default layout.

Why the lock matters: flake.nix pins by tag, and the lock turns that tag into a commit — a conversion renovate-dev-kit-lock performs unattended. A tag moved between a release and a relock would be adopted with no meaningful review, since the visible diff is a narHash, and this flake is the shell your CI runs in. Comparing the lock against the workflow pins turns that into a failed check.

A reference that is not @<sha> # vX.Y.Z fails the check rather than being skipped: one the comparison cannot read is how drift gets in unseen, and a tag-pinned reference is mutable besides. Commented-out references are ignored, so the stubs in this README can be pasted as examples.

This detects; it does not prevent. Immutable release tags on this repository would make the tag itself trustworthy.

Seeing what a flake.lock bump actually changes

A lock bump is unreviewable as a diff: one opaque rev for nixpkgs moves every package in the shell — the Go toolchain, golangci-lint, treefmt, kubectl. Renovate's weekly lockFileMaintenance refreshes every input at once and is merged by a human who has nothing to look at.

flake-closure-diff.yml builds the shell both sides of the change and posts the package-level difference to the job summary. Advisory — it reports, it never fails on a difference:

name: Flake closure diff
on:
  pull_request:
    paths: ["flake.lock", "flake.nix"]
permissions:
  contents: read
jobs:
  closure-diff:
    uses: opendefensecloud/dev-kit/.github/workflows/flake-closure-diff.yml@<40-char-sha> # <tag>
    secrets:
      cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
      cachix-signing-key: ${{ secrets.CACHIX_SIGNING_KEY }}

Pass attribute for a shell that is not devShells.x86_64-linux.default (no .# prefix — both sides are addressed as <path>#<attribute>). The base side usually comes from the cache, since it is what the last run built.

Repository settings

Run make repo-settings to reconcile your GitHub repository with the organization's standard configuration. This target is idempotent and requires the gh CLI to be authenticated.

It configures:

  • Labels — creates/updates the standard set of issue and PR labels
  • Merge strategy — merge commits only by default, configurable via REPO_ALLOW_MERGE_COMMIT, REPO_ALLOW_SQUASH_MERGE, REPO_ALLOW_REBASE_MERGE (these also restrict the ruleset's allowed merge methods; at least one must be enabled or repo-settings fails); auto-merge enabled, delete branch on merge
  • Secret scanning — enabled
  • Branch protection — a "protect-main" ruleset on the default branch (and any REPO_RULESET_BRANCHES patterns): requires PRs with 1 approval, dismisses stale reviews, requires review thread resolution, enforces commit signatures, requires approval of the latest push when REPO_REQUIRE_LAST_PUSH_APPROVAL is true, prevents direct pushes/deletions/non-fast-forwards

The repository settings are configurable via make variables (set them in your Makefile or pass them on the command line, e.g. make repo-settings REPO_STATUS_CHECKS='["CI","lint"]' REPO_RULESET_BRANCHES='["release/*"]'):

Variable Default Description
REPO_ALLOW_MERGE_COMMIT true Allow merge commits in the merge strategy
REPO_ALLOW_SQUASH_MERGE false Allow squash merging
REPO_ALLOW_REBASE_MERGE false Allow rebase merging
REPO_REQUIRE_LAST_PUSH_APPROVAL false Require the most recent push to be approved before merging
REPO_ADMIN_BYPASS true When false, org admins cannot bypass the ruleset
REPO_REQUIRED_APPROVING_REVIEW_COUNT 1 Number of approving reviews required to merge
REPO_REQUIRE_CODE_OWNER_REVIEW false Require an approving review from code owners
REPO_REQUIRE_BRANCH_UP_TO_DATE false Require branches to be up to date before merging (needs at least one REPO_STATUS_CHECKS value; repo-settings fails otherwise)
REPO_STATUS_CHECKS [] JSON array of status-check contexts that must pass (e.g. ["CI","update-action-pins / Check action pins"]); each job name is used as-is, so contexts with spaces work; the required_status_checks rule is only added when this is non-empty
REPO_RULESET_BRANCHES [] JSON array of additional branch patterns the ruleset applies to (e.g. ["release/*"]); short names are normalized to refs/heads/...; the default branch is always protected

GitHub Actions

The repo ships composite actions that consuming repositories reference directly. Pin them to a SHA with a version comment, as the action-pin check requires. The <40-char-sha> placeholders below are not copy-pasteable — substitute real digests:

Action Description
.github/actions/setup-nix Install upstream Nix and enable the shared Cachix cache
.github/actions/get-go-version Extract the Go version from flake.nix
.github/actions/diff-check Fail if a command left uncommitted changes behind

setup-nix replaces the two-step Nix installer + Cachix preamble that every job needs before it can use nix develop as its shell. Secrets are not visible inside a composite action, so the caller passes them in:

jobs:
  build:
    runs-on: ubuntu-24.04
    defaults:
      run:
        shell: nix develop --command bash -e {0}
    steps:
      - uses: actions/checkout@<40-char-sha> # v7
      - name: Set up nix
        uses: opendefensecloud/dev-kit/.github/actions/setup-nix@<40-char-sha> # <tag>
        with:
          cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
          cachix-signing-key: ${{ secrets.CACHIX_SIGNING_KEY }}
      - run: make test
Input Default Description
cachix-name opendefensecloud Cachix cache to use
cachix-auth-token '' Auth token; empty falls back to a read-only cache, as on fork PRs
cachix-signing-key '' Signing key; empty disables pushing to the cache

Governance workflows

The org-wide checks and automations are reusable workflows. A consuming repo keeps one small stub per workflow, which sets the triggers and the permissions and calls dev-kit. Pin each stub to a SHA like any other action; Renovate bumps them together with DEV_KIT_VERSION.

Workflow Stub triggers Needs
update-action-pins.yml pull_request
conventional-commits.yml pull_request (opened, edited, synchronize, reopened) .commitlintrc.yml in the repo
osv-scanner.yml pull_request, push to main, weekly optional scan-args input
issues-add-labels.yaml issues (opened, reopened)
issues-add-to-project.yml issues, pull_request (opened) ADD_TO_PROJECT_PAT secret
renovate-auto-approve.yml pull_request (opened, reopened, synchronize, labeled, unlabeled) see the header of the workflow
renovate-dev-kit-lock.yml pull_request (opened, synchronize, reopened) DEV_KIT_BOT_APP_* org secrets

No stub triggers on merge_group, as no repo uses a merge queue. A repo that turns one on must add the trigger to every stub whose check it requires, and must not require osv-scanner / scan-pr / osv-scan: that PR scan diffs against $GITHUB_BASE_REF, which is empty in a merge queue (see osv-scanner.yml).

.github/workflows/update-action-pins.yml:

name: Update Action Pins
on:
  pull_request:
permissions:
  contents: read
jobs:
  update-action-pins:
    uses: opendefensecloud/dev-kit/.github/workflows/update-action-pins.yml@<40-char-sha> # <tag>

.github/workflows/conventional-commits.yml:

name: Conventional Commits
on:
  pull_request:
    branches: ["main"]
    types: [opened, edited, synchronize, reopened]
permissions:
  pull-requests: read
jobs:
  conventional-commits:
    uses: opendefensecloud/dev-kit/.github/workflows/conventional-commits.yml@<40-char-sha> # <tag>

.github/workflows/osv-scanner.yml:

name: OSV-Scanner
on:
  pull_request:
    branches: ["main"]
  schedule:
    - cron: "12 12 * * 1"
  push:
    branches: ["main"]
permissions:
  contents: read
jobs:
  osv-scanner:
    permissions:
      actions: read
      security-events: write
      contents: read
    uses: opendefensecloud/dev-kit/.github/workflows/osv-scanner.yml@<40-char-sha> # <tag>
    # with:
    #   scan-args: |-
    #     --include-git-root
    #     --config ./.osv-scanner.toml
    #     -r
    #     ./

.github/workflows/issues-add-labels.yaml:

name: Label issues
on:
  issues:
    types: [opened, reopened]
permissions:
  contents: read
jobs:
  label:
    permissions:
      issues: write
    uses: opendefensecloud/dev-kit/.github/workflows/issues-add-labels.yaml@<40-char-sha> # <tag>

.github/workflows/issues-add-to-project.yml:

name: Add issues to project
on:
  issues:
    types: [opened]
  pull_request:
    types: [opened]
permissions:
  contents: read
jobs:
  add-to-project:
    permissions: {}
    uses: opendefensecloud/dev-kit/.github/workflows/issues-add-to-project.yml@<40-char-sha> # <tag>
    secrets:
      ADD_TO_PROJECT_PAT: ${{ secrets.ADD_TO_PROJECT_PAT }}

.github/workflows/renovate-dev-kit-lock.yml commits nix flake lock to Renovate's renovate/dev-kit PR, which bumps the flake input tag but can't update flake.lock. It commits as the org's dev-kit GitHub App, so the commit is signed and starts CI; any other PR skips it:

name: Relock dev-kit
on:
  pull_request:
    branches: ["main"]
    types: [opened, synchronize, reopened]
permissions:
  contents: read
jobs:
  relock:
    uses: opendefensecloud/dev-kit/.github/workflows/renovate-dev-kit-lock.yml@<40-char-sha> # <tag>
    secrets:
      DEV_KIT_BOT_APP_CLIENT_ID: ${{ secrets.DEV_KIT_BOT_APP_CLIENT_ID }}
      DEV_KIT_BOT_APP_PRIVATE_KEY: ${{ secrets.DEV_KIT_BOT_APP_PRIVATE_KEY }}

A job from a reusable workflow reports as <stub job> / <called job>, so REPO_STATUS_CHECKS needs the new names after switching to a stub, e.g. update-action-pins / Check action pins, conventional-commits / PR Title and conventional-commits / Commit Messages. osv-scanner nests Google's reusable workflow one level deeper, so its scan reports as osv-scanner / scan-pr / osv-scan. Update the list and rerun make repo-settings in the same change, or the old names stay required and never report.

Default git hooks

The dev shell installs the following git hooks automatically:

Hook Stage Description
fmt pre-commit Runs make fmt
lint pre-commit Runs make lint
osv-scanner pre-commit Runs make scan on dependency file changes (disabled by default — see Vulnerability scanning)
commitlint commit-msg Validates commit messages against Conventional Commits (disabled by default)

To enable the commitlint hook, set commitlint.enable = true in preCommitHooks and add a .commitlintrc.yml to the project root:

extends:
  - "@commitlint/config-conventional"
rules:
  type-enum:
    - 2
    - always
    - - feat
      - fix
      - docs
      - chore
      - refactor
      - test
      - ci
      - perf
      - revert

All hooks can be enabled or disabled per-project via preCommitHooks (see below).

Vulnerability scanning

The osv-scanner hook is disabled by default. make scan resolves every dependency against the osv.dev API, which adds roughly a minute to any commit that touches go.mod, go.sum, or requirements.txt — slow enough that it discourages small, frequent commits.

Run vulnerability scanning in CI instead, on a schedule and on every pull request. If your project has no CI-side scan, opt back in per-project:

preCommitHooks = {
  osv-scanner.enable = true;
};

Customizing the dev shell

Modify flake.nix to adjust Go version, packages, and pre-commit hooks:

{
  [...]

  outputs = { nixpkgs, flake-utils, dev-kit, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
      in
      {
        devShells.default = dev-kit.lib.mkShell {
          inherit system;
          goVersion = "1.26.2";  # Install go with a custom version
          packages = [  # add additional packages to the dev shell
            pkgs.cosign
            pkgs.trivy
          ];
          preCommitHooks = {
            custom = {  # add custom pre-commit hooks
              enable = true;
              entry = "my-custom-script";
            };
            lint = {
              enable = false;  # disable default pre-commit hooks
            };
            osv-scanner = {
              enable = true;  # enable hooks that are off by default
            };
          };
        };
      }
    );
}

Design Decisions

Why Nix?

Nix provides reproducible, declarative development environments. It ensures that every developer (and CI) operates in an identical environment, eliminating "works on my machine" issues. Nix also enables us to share modules and overlays across projects, reducing duplication and maintaining consistency.

Why Make over alternatives?

We evaluated several build tools:

  • magefile: While Go-native, it is not ideal for scripting workflows that primarily orchestrate external binaries.

  • just: Offers a modern syntax but lacks a built-in module sharing system. Migrating our Make ecosystem to just would swap one tool for another without meaningful architectural gains.

Make remains pragmatic: it is universally available and familiar to most developers. While it has its quirks — tabs for indentation, the occasional $ escape — it provides all the features we need. The curl common.mk pattern effectively gives us a module system without introducing a new dependency.

Why not devenv?

We used devenv for some time but moved away due to its dependency on an additional binary and the complexity it introduced during upgrades.

Why not Go's tool directive?

Go 1.24's tool directive in go.mod pulls tooling into the local Go module ecosystem. This often leads to dependency conflicts, as tools compiled together with the project can clash with the project's own dependencies.

Documentation

  • Nix - Package manager and dev environment
  • direnv - Environment variable loader
  • go-overlay - Go tooling for Nix

About

Development tools for opendefense.cloud projects

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages