Skip to content
This repository was archived by the owner on Jul 11, 2026. It is now read-only.

Commit e044f50

Browse files
ethanolivertroyethanolivertroy
committed
SCF API v2026.1 — automated pipeline with 249 framework crosswalks
Automated static JSON API for the Secure Controls Framework (SCF). Parses official SCF Excel releases directly from GitHub, generates 1468 controls, 33 families, and 249 bidirectional framework crosswalks. Includes GitHub Action for automatic updates when new SCF versions are released, and agent-friendly documentation following the llms.txt standard.
0 parents  commit e044f50

4,938 files changed

Lines changed: 1748554 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/release.yml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
branches: [main]
6+
paths: [".scf-version"]
7+
8+
permissions:
9+
contents: write
10+
11+
jobs:
12+
release:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout
16+
uses: actions/checkout@v4
17+
18+
- name: Read version
19+
id: version
20+
run: |
21+
VERSION=$(cat .scf-version)
22+
echo "tag=$VERSION" >> "$GITHUB_OUTPUT"
23+
echo "Version: $VERSION"
24+
25+
- name: Check if tag exists
26+
id: check
27+
env:
28+
TAG: ${{ steps.version.outputs.tag }}
29+
run: |
30+
if git rev-parse "refs/tags/$TAG" >/dev/null 2>&1; then
31+
echo "skip=true" >> "$GITHUB_OUTPUT"
32+
echo "Tag $TAG already exists, skipping."
33+
else
34+
echo "skip=false" >> "$GITHUB_OUTPUT"
35+
fi
36+
37+
- name: Get stats
38+
if: steps.check.outputs.skip != 'true'
39+
id: stats
40+
run: |
41+
CONTROLS=$(node -e "console.log(require('./docs/api/summary.json').total_controls)")
42+
FAMILIES=$(node -e "console.log(require('./docs/api/summary.json').total_families)")
43+
FRAMEWORKS=$(node -e "console.log(require('./docs/api/summary.json').crosswalk_frameworks.length)")
44+
echo "controls=$CONTROLS" >> "$GITHUB_OUTPUT"
45+
echo "families=$FAMILIES" >> "$GITHUB_OUTPUT"
46+
echo "frameworks=$FRAMEWORKS" >> "$GITHUB_OUTPUT"
47+
48+
- name: Create tag and release
49+
if: steps.check.outputs.skip != 'true'
50+
env:
51+
GH_TOKEN: ${{ github.token }}
52+
TAG: ${{ steps.version.outputs.tag }}
53+
CONTROLS: ${{ steps.stats.outputs.controls }}
54+
FAMILIES: ${{ steps.stats.outputs.families }}
55+
FRAMEWORKS: ${{ steps.stats.outputs.frameworks }}
56+
run: |
57+
git tag "$TAG"
58+
git push origin "$TAG"
59+
gh release create "$TAG" \
60+
--title "SCF $TAG" \
61+
--notes "Mirrors [SCF $TAG](https://github.com/securecontrolsframework/securecontrolsframework/releases/tag/$TAG). $CONTROLS controls, $FAMILIES families, $FRAMEWORKS framework crosswalks."

‎.github/workflows/update-scf.yml‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
name: Update SCF
2+
3+
on:
4+
schedule:
5+
# Weekly check: Monday 9:00 UTC
6+
- cron: "0 9 * * 1"
7+
workflow_dispatch:
8+
inputs:
9+
tag:
10+
description: "SCF release tag to update to (leave empty for latest)"
11+
required: false
12+
type: string
13+
force:
14+
description: "Force update even if version matches"
15+
required: false
16+
type: boolean
17+
default: false
18+
19+
permissions:
20+
contents: write
21+
pull-requests: write
22+
23+
jobs:
24+
check-and-update:
25+
runs-on: ubuntu-latest
26+
steps:
27+
- name: Checkout
28+
uses: actions/checkout@v4
29+
30+
- name: Setup Node.js
31+
uses: actions/setup-node@v4
32+
with:
33+
node-version: 20
34+
cache: npm
35+
36+
- name: Install dependencies
37+
run: npm ci
38+
39+
- name: Determine target version
40+
id: version
41+
env:
42+
GH_TOKEN: ${{ github.token }}
43+
INPUT_TAG: ${{ inputs.tag }}
44+
run: |
45+
if [ -n "$INPUT_TAG" ]; then
46+
NEW_TAG="$INPUT_TAG"
47+
else
48+
NEW_TAG=$(gh api repos/securecontrolsframework/securecontrolsframework/releases/latest --jq '.tag_name')
49+
fi
50+
CURRENT=$(cat .scf-version)
51+
echo "new_tag=$NEW_TAG" >> "$GITHUB_OUTPUT"
52+
echo "current=$CURRENT" >> "$GITHUB_OUTPUT"
53+
echo "Current: $CURRENT → Target: $NEW_TAG"
54+
55+
- name: Check if update needed
56+
id: check
57+
env:
58+
NEW_TAG: ${{ steps.version.outputs.new_tag }}
59+
CURRENT: ${{ steps.version.outputs.current }}
60+
FORCE: ${{ inputs.force }}
61+
run: |
62+
if [ "$NEW_TAG" = "$CURRENT" ] && [ "$FORCE" != "true" ]; then
63+
echo "skip=true" >> "$GITHUB_OUTPUT"
64+
echo "Already at version $CURRENT, skipping."
65+
else
66+
echo "skip=false" >> "$GITHUB_OUTPUT"
67+
fi
68+
69+
- name: Parse SCF Excel
70+
if: steps.check.outputs.skip != 'true'
71+
env:
72+
GH_TOKEN: ${{ github.token }}
73+
SCF_TAG: ${{ steps.version.outputs.new_tag }}
74+
run: node scripts/parse-scf-excel.mjs --tag "$SCF_TAG"
75+
76+
- name: Build static API
77+
if: steps.check.outputs.skip != 'true'
78+
run: npm run build
79+
80+
- name: Generate change summary
81+
if: steps.check.outputs.skip != 'true'
82+
id: summary
83+
env:
84+
NEW_TAG: ${{ steps.version.outputs.new_tag }}
85+
CURRENT: ${{ steps.version.outputs.current }}
86+
run: |
87+
CONTROLS=$(node -e "console.log(require('./docs/api/summary.json').total_controls)")
88+
FAMILIES=$(node -e "console.log(require('./docs/api/summary.json').total_families)")
89+
FRAMEWORKS=$(node -e "console.log(require('./docs/api/summary.json').crosswalk_frameworks.length)")
90+
91+
cat > /tmp/pr-body.md <<EOF
92+
## SCF Update: ${CURRENT} → ${NEW_TAG}
93+
94+
### Stats
95+
- **Controls:** ${CONTROLS}
96+
- **Families:** ${FAMILIES}
97+
- **Framework crosswalks:** ${FRAMEWORKS}
98+
99+
### Source
100+
- [SCF ${NEW_TAG} Release](https://github.com/securecontrolsframework/securecontrolsframework/releases/tag/${NEW_TAG})
101+
102+
### Generated by
103+
Automated workflow — [update-scf.yml](.github/workflows/update-scf.yml)
104+
EOF
105+
106+
- name: Remove old data files
107+
if: steps.check.outputs.skip != 'true'
108+
run: |
109+
NEW_SLUG=$(cat .scf-version | tr '.' '-')
110+
# Remove any scf-*.json that isn't the current version or crosswalks
111+
for f in data/scf-*.json; do
112+
base=$(basename "$f")
113+
if [ "$base" != "scf-${NEW_SLUG}.json" ] && [ "$base" != "scf-crosswalks.json" ]; then
114+
echo "Removing old file: $f"
115+
rm "$f"
116+
fi
117+
done
118+
# Remove defunct individual crosswalk files
119+
for f in data/hipaa-scf-crosswalk.json data/gdpr-scf-crosswalk.json data/ccpa-scf-crosswalk.json data/nis2-scf-crosswalk.json; do
120+
[ -f "$f" ] && echo "Removing: $f" && rm "$f"
121+
done
122+
123+
- name: Create Pull Request
124+
if: steps.check.outputs.skip != 'true'
125+
uses: peter-evans/create-pull-request@v7
126+
with:
127+
token: ${{ secrets.GITHUB_TOKEN }}
128+
branch: update-scf/${{ steps.version.outputs.new_tag }}
129+
title: "Update SCF to ${{ steps.version.outputs.new_tag }}"
130+
body-path: /tmp/pr-body.md
131+
commit-message: "Update SCF to ${{ steps.version.outputs.new_tag }}"
132+
labels: automated,scf-update
133+
delete-branch: true

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
node_modules
2+
.DS_Store
3+
*.xlsx
4+
.firecrawl

‎.scf-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
2026.1

‎README.md‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# SCF API
2+
3+
Source-faithful static JSON API for the [Secure Controls Framework](https://securecontrolsframework.com).
4+
5+
This repo turns the official SCF workbook into a static site under `docs/` with:
6+
7+
- SCF controls and families as JSON
8+
- Bidirectional crosswalks for every framework published in the workbook
9+
- agent-friendly discovery docs at `llms.txt`, `llms-full.txt`, and `api/docs.md`
10+
11+
The key design choice is simple: this API follows the source material, not the old hand-curated slugs from earlier versions of this repo. Framework IDs, filenames, and mapped control IDs are emitted from the SCF workbook and related source data as-is.
12+
13+
## For Agents
14+
15+
Start here:
16+
17+
- `llms.txt` — short index for discovery
18+
- `llms-full.txt` — exhaustive framework catalog
19+
- `api/docs.md` — full endpoint reference, examples, and workflows
20+
21+
The docs are generated alongside the JSON API so agents can discover valid routes before fetching large files.
22+
23+
## Files
24+
25+
| Path | What it contains |
26+
|------|-----------------|
27+
| `api/summary.json` | Version info, control counts, family breakdown, framework coverage, weight distribution |
28+
| `api/controls.json` | All controls with crosswalk mappings |
29+
| `api/controls/{ID}.json` | Single control, including all mapped frameworks |
30+
| `api/families.json` | Family index with counts |
31+
| `api/families/{CODE}.json` | Single family with all of its controls |
32+
| `api/crosswalks.json` | Framework index with exact source-derived framework IDs |
33+
| `api/crosswalks/{FRAMEWORK_ID}.json` | Full bidirectional crosswalk for one framework |
34+
| `api/docs.md` | Agent-friendly API reference in markdown |
35+
| `llms.txt` | Short machine-readable index |
36+
| `llms-full.txt` | Full machine-readable framework catalog |
37+
38+
## Source-Faithful Contract
39+
40+
- Framework IDs come from the SCF workbook's source metadata.
41+
- Framework control IDs are published exactly as they appear in the workbook.
42+
- Old compatibility aliases are intentionally not preserved.
43+
- If you need a valid framework route, fetch `api/crosswalks.json` first and use the `framework_id` values it returns.
44+
45+
That keeps the API aligned with SCF releases and avoids guessing when the workbook changes.
46+
47+
## Examples
48+
49+
```bash
50+
# Fetch the generated agent docs
51+
curl https://ethanolivertroy.github.io/scf-api/api/docs.md
52+
53+
# Fetch the full controls dataset
54+
curl https://ethanolivertroy.github.io/scf-api/api/controls.json
55+
56+
# Discover valid framework IDs first
57+
curl https://ethanolivertroy.github.io/scf-api/api/crosswalks.json
58+
59+
# Fetch one source-faithful framework crosswalk
60+
curl https://ethanolivertroy.github.io/scf-api/api/crosswalks/general-nist-800-53-r5-2.json
61+
62+
# Fetch a single control with all of its crosswalks
63+
curl https://ethanolivertroy.github.io/scf-api/api/controls/GOV-01.json
64+
```
65+
66+
## Rebuilding
67+
68+
Install dependencies:
69+
70+
```bash
71+
npm ci
72+
```
73+
74+
Build the static site from the checked-in data:
75+
76+
```bash
77+
npm run build
78+
```
79+
80+
Parse a local SCF workbook into `data/` and update `.scf-version`:
81+
82+
```bash
83+
npm run parse -- --file path/to/secure-controls-framework.xlsx
84+
```
85+
86+
Or download a specific release tag from GitHub first:
87+
88+
```bash
89+
npm run parse -- --tag 2026.1
90+
```
91+
92+
The parser writes:
93+
94+
- `data/scf-{version}.json`
95+
- `data/scf-crosswalks.json`
96+
- `.scf-version`
97+
98+
Then `npm run build` regenerates `docs/`.
99+
100+
## Automation
101+
102+
The repo includes `.github/workflows/update-scf.yml`, which can:
103+
104+
- check for a new SCF release on a schedule
105+
- parse the latest workbook
106+
- rebuild the static site
107+
- open a PR with the updated generated files
108+
109+
## Note on CORS
110+
111+
GitHub Pages does not set CORS headers. CLI tools, MCP servers, and server-side code can fetch these files directly. Browser-based JavaScript on another origin will need a proxy or a different host.
112+
113+
## Data License
114+
115+
SCF data is from [securecontrolsframework.com](https://securecontrolsframework.com), licensed under CC BY-ND.

0 commit comments

Comments
 (0)