Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
},
"metadata": {
"description": "Construction skills for AI agents: document-based estimating and project controls workflows. No Buildr account required.",
"version": "0.6.0"
"version": "0.7.0"
},
"plugins": [
{
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,5 @@ jobs:
run: node --test evals/tasks/workforce-planning/staffing-core.test.mjs
- name: Test bid leveling calculations and tab build
run: node --test evals/tasks/bid-leveling/leveling-core.test.mjs
- name: Test RFP intake gates, scoring, and brief
run: node --test evals/tasks/rfp-intake/intake.test.mjs
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ text and inputs as attachments; script or PDF workflows still require a runtime.
| [bid-leveling](skills/bid-leveling) | Preconstruction, Estimating | Level sub bids on bid day. See every scope gap, accept or enter plugs, and find out who is really low once the scope matches. |
| [drawing-scope-extraction](skills/drawing-scope-extraction) | Preconstruction, Estimating | Turn drawing sheets into a cited scope list by CSI division, with exclusions, open questions, and review coverage. |
| [proposal-builder](skills/proposal-builder) | Business development, Estimating | Build polished construction proposal PDFs with pdfcn, sourced scope, pricing and qualifications. |
| [rfp-intake](skills/rfp-intake) | Business development, Preconstruction | Turn a solicitation and its addenda into a cited bid/no-bid review with current requirements, risks, and a conditional recommendation. |
| [rfp-intake](skills/rfp-intake) | Business development, Preconstruction | Prep the go/no-go meeting. Check the must-pass gates, score the opportunity on your criteria, and get a cited brief with dates, addenda changes, risks, and questions. |
| [workforce-planning](skills/workforce-planning) | Workforce, Operations | Prep the staffing meeting: who is overloaded, which jobs have empty seats, who rolls off when, and what changes if you win the pursuits you are chasing. |

The catalog contains five skills. See [ROADMAP.md](ROADMAP.md) for what is next.
Expand Down
73 changes: 54 additions & 19 deletions catalog.json

Large diffs are not rendered by default.

75 changes: 74 additions & 1 deletion evals/reviews/rfp-intake/REPORT.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,77 @@
# RFP intake takeover report
# RFP intake verification

## Revision 2.0.0: scripted go/no-go brief

Previous revision: 1.1.0, a prompt-only review written straight to Markdown
from a default checklist. This revision keeps the job (prep the bid/no-bid
meeting for one opportunity) and makes the decision checkable.

### What changed

- **Company profile.** `company-profile.json` is reused across runs. It holds
licenses, bonding and insurance limits, days needed to bid, terms the
company won't accept, scoring context, and the go/no-go criteria, weights,
and thresholds. The first run asks for it once. Missing facts become
Unknown, never guesses.
- **Gates.** `scripts/intake.mjs` checks:
- licenses, where a license needed at bid and not held fails, and one
needed before contract is a condition
- single and aggregate bonding
- insurance limits
- mandatory meetings already passed
- days to bid
- disqualifying terms

Each gate is pass, fail, or unknown.
- **Score.** Weighted 0 to 5 criteria scaled to 100. Unknown criteria are left
out, and their share of the weight is reported.
- **Recommendation.**
- Any failed gate means No-go.
- Unknown gates, too much unknown weight, or a score between the thresholds
mean Go with conditions, each condition written as an action.
- Otherwise the score decides Go or No-go.
- **Citations are validated.** Every fact cites `{ doc, page }`. The script
rejects pages beyond a document's page count, and citations to documents
that were referenced but never supplied.
- The brief keeps the 1.1.0 headings the Brannock verifier checks, and adds
Gates, "This week", and Changed by addenda.
- `references/intake-checklist.md` is replaced by
`references/data-contract.md` and `references/reading-solicitations.md`.

### Sample

Cedar Hollow Public Schools, Prairie View Elementary addition and renovation,
an $18.2M hard bid. The package ships as three PDFs: an 8-page ITB and two
addenda. Planted problems:

- a mandatory pre-bid meeting three days out
- a city license required before contract that the company doesn't hold
- pollution liability added by Addendum 2 and missing from the profile
- a bond at 91% of the single-job limit
- liquidated damages against a renovation window that ends on the
Substantial Completion date
- the hazardous materials survey referenced but not supplied

Result: Go with conditions, a score of 63 against a go line of 65, with 20%
of the weight unknown.

### Verification

- `node --test evals/tasks/rfp-intake/intake.test.mjs`: 11 tests, in CI. They
include an exact match of the bundled brief against a fresh build, each of
the 7 gate failures forcing No-go, unknowns forcing conditions, score
thresholds, default criteria, and citation validation.
- Mutation check: 10 deliberate breaks each failed at least one test.
- The sample brief passes the Brannock task's header and section checks.

### Not yet tested

A fresh agent extracting a solicitation it hasn't seen, including physical
page numbers on PDFs whose printed numbers differ.

---

# Revision 1.1.0: takeover report

RFP intake is revised to 1.1.0. Changes are local and limited to
`skills/rfp-intake/`, `evals/tasks/rfp-intake/`, and this review directory.
Expand Down
6 changes: 3 additions & 3 deletions evals/tasks/rfp-intake/brannock-fire-station-4/Task.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ the agent image or workspace.
the risks that bite, each cited to the section it came from, ending in a
scorecard and a recommendation the user can act on.
- Capability being tested: the `rfp-intake` skill's end-to-end workflow (read
the whole package first, apply the default go/no-go criteria in
`references/intake-checklist.md`, cover its output content, cite every fact,
the whole package first, apply go/no-go gates and criteria per
`references/data-contract.md`, cover the brief's content, cite every fact,
recommend rather than decide, and stay out of legal advice and estimating).
- Why this case matters: intake is the skill's core job, and the failure mode
that matters in practice is a confident summary that drops half the
requirements or invents a date. Both are checkable without a judge.
- Evidence: `skills/rfp-intake/SKILL.md`, `references/intake-checklist.md`, and
- Evidence: `skills/rfp-intake/SKILL.md`, `references/reading-solicitations.md`, and
the shipped sample. The shipped sample cannot be the eval input because the
skill folder contains its expected output; this solicitation is new.
- Difference from existing Tasks: `pay-app-review/harborview-app3` scores
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""Format: the review follows the output template in the skill's
references/intake-checklist.md."""
"""Format: the review has the headings of the skill's go/no-go brief
(scripts/intake.mjs)."""

from __future__ import annotations

Expand Down
190 changes: 190 additions & 0 deletions evals/tasks/rfp-intake/intake.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { analyze, daysBetween, formatDate, money, renderBrief, validate } from "../../../skills/rfp-intake/scripts/intake.mjs";

const skill = fileURLToPath(new URL("../../../skills/rfp-intake/", import.meta.url));
const script = join(skill, "scripts/intake.mjs");
const load = (name) => JSON.parse(readFileSync(join(skill, "samples", name), "utf8"));
const sample = () => ({ data: load("intake-data.json"), profile: load("company-profile.json") });
const gate = (a, key) => a.gates.find((g) => g.key === key);
const run = ({ data, profile }) => {
validate(data, profile);
return analyze(data, profile);
};

// A small opportunity that passes everything, for rule-level tests.
function clean() {
const cite = (page) => ({ doc: "rfp", page });
const data = {
opportunity: { name: "Clinic", owner: "County", asOf: "2027-01-04", estimatedValue: 500000000, estimatedValueCite: cite(1) },
documents: [{ id: "rfp", title: "RFP", supplied: true, pages: 3 }],
dates: [
{ key: "prebid", label: "Pre-bid", date: "2027-01-08", mandatory: true, cite: cite(1) },
{ key: "bid_due", label: "Bids due", date: "2027-02-01", time: "14:00", tz: "ET", cite: cite(1) },
],
bonds: { text: "100% P&P", performancePercent: 100, cite: cite(2) },
insurance: [{ key: "umbrella", label: "Umbrella", required: 500000000, cite: cite(2) }],
licenses: [{ key: "state-gc", label: "State GC", when: "bid", cite: cite(2) }],
terms: [],
criteria: [{ key: "fit", score: 4, reason: "Core sector.", cite: cite(3) }, { key: "risk", score: 4, reason: "Standard terms.", cite: cite(3) }],
};
const profile = {
company: "GC",
licenses: [{ key: "state-gc", label: "State GC" }],
bonding: { singleLimit: 2000000000, aggregateLimit: 6000000000, currentBonded: 1000000000 },
insurance: { umbrella: 1000000000 },
minDaysToBid: 14,
disqualifyingTerms: [{ key: "no_damages_for_delay", label: "No damages for delay" }],
criteria: [{ key: "fit", label: "Fit", weight: 60 }, { key: "risk", label: "Risk", weight: 40 }],
thresholds: { go: 65, noGo: 45, maxUnknownWeight: 25 },
};
return { data, profile };
}

test("bundled brief matches a fresh build exactly", () => {
const dir = mkdtempSync(join(tmpdir(), "intake-"));
const out = spawnSync(process.execPath, [script, join(skill, "samples/intake-data.json"), join(skill, "samples/company-profile.json"), "--brief", join(dir, "brief.md")], { encoding: "utf8" });
assert.equal(out.status, 0, out.stderr);
assert.equal(readFileSync(join(dir, "brief.md"), "utf8"), readFileSync(join(skill, "samples/output-intake-brief.md"), "utf8"));
});

test("sample: go with conditions for the license, pollution cover, and a middling score", () => {
const a = run(sample());
assert.equal(a.verdict, "Go with conditions");
assert.equal(a.score, 63);
assert.equal(a.unknownWeight, 20);
assert.equal(gate(a, "license_city-cedar-hollow-class-a").status, "unknown");
assert.equal(gate(a, "insurance").status, "unknown");
assert.match(gate(a, "insurance").detail, /pollution liability/);
assert.equal(gate(a, "bonding").status, "pass");
assert.match(gate(a, "bonding").detail, /91% of the \$20,000,000 single-job limit/);
assert.equal(a.risks[0].title, "Bond close to the single-job limit");
assert.equal(gate(a, "mandatory_prebid").status, "pass");
assert.deepEqual(a.soon.map((d) => d.key), ["prebid"]);
assert.equal(a.conditions.length, 3);
});

test("a clean opportunity is a go", () => {
const a = run(clean());
assert.ok(a.gates.every((g) => g.status === "pass"), JSON.stringify(a.gates));
assert.equal(a.score, 80);
assert.equal(a.verdict, "Go");
});

test("each failed gate forces no-go regardless of score", () => {
const cases = {
"license at bid": (x) => (x.profile.licenses = []),
"over single bond": (x) => (x.data.opportunity.estimatedValue = 2500000000),
"over aggregate bond": (x) => (x.profile.bonding.currentBonded = 5600000000),
"short insurance": (x) => (x.profile.insurance.umbrella = 200000000),
"missed meeting": (x) => (x.data.opportunity.asOf = "2027-01-09"),
"too little time": (x) => (x.profile.minDaysToBid = 30),
"disqualifying term": (x) => x.data.terms.push({ key: "no_damages_for_delay", label: "No damages for delay", cite: { doc: "rfp", page: 2 } }),
};
for (const [name, mutate] of Object.entries(cases)) {
const x = clean();
mutate(x);
const a = run(x);
assert.equal(a.verdict, "No-go", name);
assert.ok(a.gates.some((g) => g.status === "fail"), name);
}
});

test("unknown company facts make the recommendation conditional, never a pass", () => {
const x = clean();
delete x.profile.licenses;
delete x.profile.minDaysToBid;
x.profile.insurance = {};
const a = run(x);
assert.equal(a.verdict, "Go with conditions");
assert.deepEqual(a.gates.filter((g) => g.status === "unknown").map((g) => g.key).sort(), ["due_date", "insurance", "license_state-gc"]);
assert.equal(a.conditions.length, 3);
});

test("a license needed only at contract is a condition, not a failure", () => {
const x = clean();
x.data.licenses[0].when = "contract";
x.profile.licenses = [];
const a = run(x);
assert.equal(gate(a, "license_state-gc").status, "unknown");
assert.equal(a.verdict, "Go with conditions");
});

test("the score uses weights, leaves unknowns out, and respects thresholds", () => {
const x = clean();
x.data.criteria[0].score = 2; // 2*60 + 4*40 = 280 of 500
let a = run(x);
assert.equal(a.score, 56);
assert.equal(a.verdict, "Go with conditions");
x.data.criteria[0].score = 1; // 220 of 500
a = run(x);
assert.equal(a.score, 44);
assert.equal(a.verdict, "No-go");
const y = clean();
y.data.criteria[1] = { key: "risk", score: null, reason: "Contract not supplied." };
a = run(y);
assert.equal(a.score, 80);
assert.equal(a.unknownWeight, 40);
assert.equal(a.verdict, "Go with conditions");
assert.match(a.conditions.join(), /40% of the criteria weight is unknown/);
});

test("default criteria apply when the profile has none, and the brief says so", () => {
const x = clean();
delete x.profile.criteria;
x.data.criteria = ["owner_relationship", "delivery_contract", "schedule", "scope_fit", "size_fit", "competition", "risk_allocation", "capacity"].map((key) => ({ key, score: 3, reason: "Neutral.", cite: { doc: "rfp", page: 3 } }));
const a = run(x);
assert.equal(a.score, 60);
assert.ok(a.usedDefaults);
assert.match(renderBrief(x.data, x.profile, a), /the skill's defaults/);
});

test("citations must point at supplied pages that exist", () => {
const bad = (mutate, pattern) => {
const x = clean();
mutate(x);
assert.throws(() => validate(x.data, x.profile), pattern);
};
bad((x) => (x.data.dates[1].cite.page = 9), /page 9 is not in rfp \(3 pages\)/);
bad((x) => (x.data.dates[1].cite = { doc: "manual", page: 1 }), /cites document "manual", which is not in documents/);
bad((x) => {
x.data.documents.push({ id: "manual", title: "Project Manual", supplied: false });
x.data.insurance[0].cite = { doc: "manual", page: 4 };
}, /which was not supplied/);
bad((x) => (x.data.dates = x.data.dates.slice(0, 1)), /must include key "bid_due"/);
bad((x) => (x.data.criteria = x.data.criteria.slice(0, 1)), /"risk" \(Risk\) is not scored/);
bad((x) => (x.data.criteria[0].score = 6), /integer 0 to 5/);
bad((x) => delete x.data.criteria[0].cite, /needs a cite from the solicitation or a profileField/);
bad((x) => (x.data.opportunity.estimatedValue = 5e8 + 0.5), /integer cents/);
});

test("formatting helpers", () => {
assert.equal(money(1820000000), "$18,200,000");
assert.equal(money(null), "not stated");
assert.equal(formatDate("2026-10-29", "14:00", "CT"), "Thu Oct 29, 2026, 2:00 p.m. CT");
assert.equal(formatDate("2026-10-08", "10:00"), "Thu Oct 8, 2026, 10:00 a.m.");
assert.equal(formatDate("2026-10-08", "00:30"), "Thu Oct 8, 2026, 12:30 a.m.");
assert.equal(daysBetween("2026-10-05", "2026-10-29"), 24);
assert.equal(daysBetween("2026-12-31", "2027-01-01"), 1);
});

test("CLI: --json, clear errors, and usage", () => {
const data = join(skill, "samples/intake-data.json");
const profile = join(skill, "samples/company-profile.json");
const json = spawnSync(process.execPath, [script, data, profile, "--json"], { encoding: "utf8" });
assert.equal(json.status, 0, json.stderr);
assert.equal(JSON.parse(json.stdout).verdict, "Go with conditions");
const dir = mkdtempSync(join(tmpdir(), "intake-"));
const broken = load("intake-data.json");
broken.risks[0].cite.page = 12;
writeFileSync(join(dir, "broken.json"), JSON.stringify(broken));
const bad = spawnSync(process.execPath, [script, join(dir, "broken.json"), profile], { encoding: "utf8" });
assert.equal(bad.status, 1);
assert.match(bad.stderr, /page 12 is not in itb \(8 pages\)/);
assert.equal(spawnSync(process.execPath, [script], { encoding: "utf8" }).status, 2);
});
Loading
Loading