diff --git a/hypercerts-api/README.md b/hypercerts-api/README.md index fa3f26fa..e5f86e99 100644 --- a/hypercerts-api/README.md +++ b/hypercerts-api/README.md @@ -2,7 +2,7 @@ This directory contains the tooling for building and testing a Hypercerts XRPC API on HappyView. It provides a reusable installer for HappyView admin assets (Lexicons and Lua scripts), pinned Hypercerts Lexicon dependencies, and test fixtures for checking record behavior. The installer lets API modules declare their assets together, install missing assets in dependency order, and refuse to overwrite assets that differ from what is already installed. -**Current status:** This branch contains the installer and offline test tooling, not an installable API. There is no production `manifest.json` or Lua handler bundle here yet. You can run the unit tests, but do not run `tooling/installer.js` against a HappyView target from this branch. +**Current status:** The root `manifest.json` includes the location API bundle. Build its Lua handlers and run the offline checks before installing it on an approved HappyView target. ## Get started @@ -10,12 +10,31 @@ From `hypercerts-api/`: ```sh pnpm install --frozen-lockfile +pnpm build:lua pnpm check ``` -`pnpm check` runs JavaScript and Lua lint, a strict `checkJs` typecheck of the installer, then the offline unit tests. Run `pnpm run typecheck` to run just the typecheck. TypeScript is a development-time checker only: the installer remains JavaScript and runs directly with Node, with no transpilation. These checks do not require a running HappyView instance or database. The installable API bundle and its endpoint contract tests must be supplied separately before you can install or exercise API endpoints. +These checks run offline; they do not require a running HappyView instance or database. `build:lua` builds the standalone location handlers in `lua/endpoints/`. `pnpm check` runs JavaScript and Lua lint, a strict `checkJs` typecheck of the installer, then the offline unit tests; run `pnpm run typecheck` to run just the typecheck. TypeScript is a development-time checker only: the installer remains JavaScript and runs directly with Node, with no transpilation. The bundle combines local API Lexicons and scripts with schemas from the pinned `@hypercerts-org/lexicon` package. HTTP contract tests require an installed bundle and a separately approved disposable target. -ESLint is installed with the package dependencies. It uses ESLint's recommended checks plus strict equality, no implicit coercion, no shadowed names, no reassigned parameters, no `var`, and `const` where possible. Console output is allowed only in CLI tooling. When a branch contains Lua scripts, Luacheck checks the generated handlers in `lua/endpoints/`; it rejects unknown globals and unused arguments, allowing only HappyView's `db`, `json`, `params`, `toarray`, and `handle`. It targets Lua 5.4 and skips line-length checks for long SQL expressions. Install Lua 5.4 and LuaRocks, then run `luarocks --lua-version=5.4 --local install luacheck 1.2.0` to enable that check locally. The lint runner also finds the default `~/.luarocks/bin` install if it is not on `PATH`. On this foundation-only branch, which has no Lua files, the Lua check reports that it is skipped. If Lua sources exist but generated handlers are missing, run `pnpm build:lua` first. +ESLint is installed with the package dependencies. It uses ESLint's recommended checks plus strict equality, no implicit coercion, no shadowed names, no reassigned parameters, no `var`, and `const` where possible. Console output is allowed only in CLI tooling. When a branch contains Lua scripts, Luacheck checks the generated handlers in `lua/endpoints/`; it rejects unknown globals and unused arguments, allowing only HappyView's `db`, `json`, `params`, `toarray`, and `handle`. It targets Lua 5.4 and skips line-length checks for long SQL expressions. Install Lua 5.4 and LuaRocks, then run `luarocks --lua-version=5.4 --local install luacheck 1.2.0` to enable that check locally. The lint runner also finds the default `~/.luarocks/bin` install if it is not on `PATH`. Branches without Lua files skip Luacheck. If Lua sources exist but generated handlers are missing, run `pnpm build:lua` first. + +## Bootstrap a running HappyView instance + +After those offline checks pass, run this from `hypercerts-api/` to install the bundle on an approved, running HappyView instance. Unlike the checks above, this command contacts the instance and uploads assets. + +For an interactive run, start the installer and enter the requested URL and admin token when prompted. Token input is hidden. Any nonblank values already set in the environment are used, and the installer prompts only for missing values. + +```sh +pnpm install:api +``` + +For a noninteractive run, provide both values in the environment: + +```sh +HAPPYVIEW_BASE_URL='https://your-happyview.example' \ +HAPPYVIEW_ADMIN_TOKEN='' \ +pnpm install:api +``` ## How an API bundle is installed @@ -69,12 +88,16 @@ The root manifest lists module manifests once: Each module manifest declares `{ "assets": [...] }`. Assets have an `id`, `kind` (`lexicon` or `script`), `config`, and optional `dependsOn` asset IDs. Lexicons point to a `packagePath` in `@hypercerts-org/lexicon` or a local `path`; scripts use a local `path`. Local paths are relative to the **module manifest that declares them**. Declare shared assets in one module and reference their IDs from dependent modules. The installer rejects duplicate IDs, missing dependencies, cycles, and invalid source files before making admin requests. -With a complete bundle and an explicitly approved HappyView target, the entry point is `node tooling/installer.js`. Set `HAPPYVIEW_BASE_URL` and `HAPPYVIEW_ADMIN_TOKEN` in the environment. The installer sends `Authorization: Bearer `; session-cookie authentication is not supported. Remote targets must use HTTPS; HTTP is allowed only on `localhost`, `127.0.0.1`, or `::1`. URL credentials and HTTP redirects are rejected. +The root manifest's `validationLexicons` is the complete local schema closure used by Lexicon validation; it does not define what the installer uploads. Only assets listed in a module's `assets` array are deployed. The location module intentionally leaves package-backed support schemas in the validation closure without deploying them. Removing an asset from a manifest affects future installer runs only; the installer does not unregister assets already on the HappyView instance. + +The bootstrap command above should target only an explicitly approved HappyView instance. It sends `Authorization: Bearer `; session-cookie authentication is not supported. -The admin API key must have `lexicons:read` and `lexicons:create` for lexicon assets. If the bundle includes scripts, it also needs `scripts:read` and `scripts:manage`. If a manifest requests backfill for a new record lexicon, `backfill:create` is additionally needed to start that job; without it, the lexicon is still uploaded but no backfill starts. Do not use the installer until the bundle includes its production manifests, handlers, and domain-specific completeness tests. +The token must have `lexicons:read` and `lexicons:create` for lexicon assets. If the bundle includes scripts, it also needs `scripts:read` and `scripts:manage`. If a manifest requests backfill for a new record lexicon, `backfill:create` is additionally needed to start that job; without it, the lexicon is still uploaded but no backfill starts. Remote targets must use HTTPS; HTTP is allowed only on `localhost`, `127.0.0.1`, or `::1`. URL credentials and HTTP redirects are rejected. Resolve installed-asset conflicts manually. The location handlers and fixtures require PostgreSQL. ## Test fixtures (disposable databases only) The fixture tools seed records directly into PostgreSQL, bypassing HappyView ingestion. Use them **only with a separately approved, disposable loopback test database**, never persistent data. The `seed:test` and `seed:bad-dates` scripts require `HAPPYVIEW_DISPOSABLE_TEST_TARGET=YES`, a loopback `PGHOST`, a `PGDATABASE` name containing a `test` marker, and an absolute `PSQL_PATH` to a trusted `psql` executable. Standard `PGPORT` and `PGUSER` can select the test instance and user. The normal `seed:test` command seeds location, profile, and organization examples by default; `seed:bad-dates` seeds only location examples by default. Both tools also accept supplied record rows through `buildSeedInput` and `buildBadDateSeedInput` in `tooling/seed.js`. Fixture CIDs are computed from stored record JSON with `@atcute/cbor` and `@atcute/cid`. If you seeded an older version of the blob-backed fixture, reseed the **disposable test database** before comparing CIDs: the earlier `jsonToLex` conversion produced a different CID. + +After confirming the approved disposable PostgreSQL database is the one used by the installed HappyView instance, run `pnpm seed:test` and `pnpm test:contracts` with `HAPPYVIEW_BASE_URL` set to that instance. Optionally run `pnpm seed:bad-dates` and then `pnpm test:bad-dates` on the same target. These commands are not part of the offline unit suite; seeding bypasses ingestion and tests only the read path. diff --git a/hypercerts-api/lexicons/app.certified.location.getLocation.json b/hypercerts-api/lexicons/app.certified.location.getLocation.json new file mode 100644 index 00000000..6655b9f9 --- /dev/null +++ b/hypercerts-api/lexicons/app.certified.location.getLocation.json @@ -0,0 +1,37 @@ +{ + "lexicon": 1, + "id": "app.certified.location.getLocation", + "defs": { + "main": { + "type": "query", + "description": "Public lookup of one indexed location record by full AT-URI. Authentication is not required.", + "parameters": { + "type": "params", + "required": ["uri"], + "properties": { + "uri": { "type": "string", "format": "at-uri", "description": "Full record AT-URI using a DID authority." } + } + }, + "output": { "encoding": "application/json", "schema": { "type": "ref", "ref": "#output" } }, + "errors": [{ "name": "RecordNotFound", "description": "No indexed location exists at this AT-URI." }] + }, + "output": { + "type": "object", + "required": ["location"], + "properties": { "location": { "type": "ref", "ref": "#locationView" } } + }, + "locationView": { + "type": "object", + "description": "Location record with full indexed metadata, original payload, and hydrated author.", + "required": ["uri", "cid", "indexedAt", "did", "author", "record"], + "properties": { + "uri": { "type": "string", "format": "at-uri" }, + "cid": { "type": "string", "format": "cid" }, + "indexedAt": { "type": "string", "format": "datetime" }, + "did": { "type": "string", "format": "did" }, + "author": { "type": "ref", "ref": "org.hypercerts.api.defs#actorView" }, + "record": { "type": "ref", "ref": "app.certified.location" } + } + } + } +} diff --git a/hypercerts-api/lexicons/app.certified.location.listLocations.json b/hypercerts-api/lexicons/app.certified.location.listLocations.json new file mode 100644 index 00000000..25d9b237 --- /dev/null +++ b/hypercerts-api/lexicons/app.certified.location.listLocations.json @@ -0,0 +1,31 @@ +{ + "lexicon": 1, + "id": "app.certified.location.listLocations", + "defs": { + "main": { + "type": "query", + "description": "Public listing of indexed location records. Authentication is not required. Array filters use repeated unbracketed keys, maximum 100 supplied values before deduplication; omission is unrestricted. The handler applies limit=25 and sortDirection=desc when omitted; limit is 1..100. Results sort by a valid zoned createdAt, otherwise indexedAt (or the stored row creation time when indexedAt is absent), then URI, both in the requested direction; the opaque direction-bound cursor captures that database timestamp at UTC microsecond precision, with no snapshot guarantee. Keep other parameters unchanged between pages. Unknown parameters and repeated scalar keys are rejected.", + "parameters": { + "type": "params", + "properties": { + "authors": { "type": "array", "maxLength": 100, "items": { "type": "string", "format": "did" } }, + "uris": { "type": "array", "description": "Full record AT-URIs using DID authorities.", "maxLength": 100, "items": { "type": "string", "format": "at-uri" } }, + "locationTypes": { "type": "array", "description": "Exact open-string match against locationType; empty strings are values, not sentinels.", "maxLength": 100, "items": { "type": "string", "maxLength": 20 } }, + "limit": { "type": "integer", "description": "Page size; handler default is 25.", "minimum": 1, "maximum": 100 }, + "cursor": { "type": "string", "description": "Opaque versioned keyset cursor bound to sortDirection." }, + "sortDirection": { "type": "string", "description": "Sort direction; handler default is desc.", "enum": ["asc", "desc"] } + } + }, + "output": { "encoding": "application/json", "schema": { "type": "ref", "ref": "#output" } }, + "errors": [{ "name": "InvalidRequest", "description": "A query parameter is invalid, repeated where scalar, or unknown." }] + }, + "output": { + "type": "object", + "required": ["locations"], + "properties": { + "locations": { "type": "array", "items": { "type": "ref", "ref": "app.certified.location.getLocation#locationView" } }, + "cursor": { "type": "string"} + } + } + } +} diff --git a/hypercerts-api/lexicons/org.hypercerts.api.defs.json b/hypercerts-api/lexicons/org.hypercerts.api.defs.json new file mode 100644 index 00000000..df60b2a4 --- /dev/null +++ b/hypercerts-api/lexicons/org.hypercerts.api.defs.json @@ -0,0 +1,42 @@ +{ + "lexicon": 1, + "id": "org.hypercerts.api.defs", + "description": "Shared views for Hypercerts API read models.", + "defs": { + "profileView": { + "type": "object", + "description": "Profile record view preserving the full indexed record.", + "required": ["uri", "cid", "indexedAt", "did", "record"], + "properties": { + "uri": { "type": "string", "format": "at-uri" }, + "cid": { "type": "string", "format": "cid" }, + "indexedAt": { "type": "string", "format": "datetime" }, + "did": { "type": "string", "format": "did" }, + "record": { "type": "ref", "ref": "app.certified.actor.profile" } + } + }, + "organizationView": { + "type": "object", + "description": "Raw organization sidecar record view preserving the full indexed record.", + "required": ["uri", "cid", "indexedAt", "did", "record"], + "properties": { + "uri": { "type": "string", "format": "at-uri" }, + "cid": { "type": "string", "format": "cid" }, + "indexedAt": { "type": "string", "format": "datetime" }, + "did": { "type": "string", "format": "did" }, + "record": { "type": "ref", "ref": "app.certified.actor.organization" } + } + }, + "actorView": { + "type": "object", + "description": "Actor DID with nullable profile and raw organization sidecar views.", + "required": ["did", "profile", "organization"], + "nullable": ["profile", "organization"], + "properties": { + "did": { "type": "string", "format": "did" }, + "profile": { "type": "ref", "ref": "#profileView" }, + "organization": { "type": "ref", "ref": "#organizationView" } + } + } + } +} diff --git a/hypercerts-api/lua/endpoints/getLocation.lua b/hypercerts-api/lua/endpoints/getLocation.lua new file mode 100644 index 00000000..d98d939d --- /dev/null +++ b/hypercerts-api/lua/endpoints/getLocation.lua @@ -0,0 +1,104 @@ +local COLLECTION = "app.certified.location" +local PROFILE = "app.certified.actor.profile" +local ORGANIZATION = "app.certified.actor.organization" +local NULL = json.decode("null") + +local function invalid(message) + error("InvalidRequest: " .. message, 0) +end + +local function keys_only(values, allowed) + for key in pairs(values) do + if not allowed[key] then invalid("unknown query parameter") end + end +end + +local function scalar(params, key) + local value = params[key] + if value == nil then return nil end + if type(value) ~= "string" and type(value) ~= "number" then + invalid(key .. " must occur once") + end + return tostring(value) +end + +local function valid_did(value) + if #value > 2048 then return false end + local method, specific = value:match("^did:([a-z]+):(.+)$") + if not method or not specific or specific:sub(-1) == ":" or specific:sub(-1) == "%" + or value:find("[^%w%.:_%%%-]") then return false end + return true +end + +local function valid_record_key(value) + return #value >= 1 and #value <= 512 and value ~= "." and value ~= ".." + and not value:find("[^%w_~%.:%-]") +end + +local function valid_uri(value) + if value:find("[?#]") then return false end + local authority, collection, rkey = value:match("^at://([^/]+)/([^/]+)/([^/]+)$") + return authority ~= nil and valid_did(authority) and collection == COLLECTION and valid_record_key(rkey) +end + +local function query(sql, values) + local ok, result = pcall(db.raw, sql, values) + if not ok then error("LocationQueryFailed: location lookup failed", 0) end + return result +end + +local function row_view(row) + local record = json.decode(row.record) + return { + uri = row.uri, cid = row.cid, indexedAt = row.indexed_at, did = row.did, + record = record, + } +end + +local function hydrate(views) + if #views == 0 then return end + local dids, seen = {}, {} + for _, view in ipairs(views) do + if not seen[view.did] then seen[view.did] = true; dids[#dids + 1] = view.did end + end + local profiles, organizations = {}, {} + local function load(collection, target) + if #dids == 0 then return end + local params, marks = {}, {} + params[1] = collection + for _, did in ipairs(dids) do params[#params + 1] = did; marks[#marks + 1] = "$" .. #params end + local rows = query("SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record FROM happyview_records WHERE collection = $1 AND rkey = 'self' AND did IN (" .. table.concat(marks, ",") .. ")", params) + for _, row in ipairs(rows) do target[row.did] = row end + end + load(PROFILE, profiles) + load(ORGANIZATION, organizations) + for _, view in ipairs(views) do + local profile, organization = profiles[view.did], organizations[view.did] + local author = { did = view.did, profile = NULL, organization = NULL } + if profile then author.profile = row_view(profile) end + if organization then author.organization = row_view(organization) end + view.author = author + end +end + +local function query_location(uri) + if db.backend() ~= "postgres" then error("LocationQueryFailed: location API requires PostgreSQL", 0) end + local rows = query("SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record FROM happyview_records WHERE collection = $1 AND uri = $2 LIMIT 1", { COLLECTION, uri }) + local views = {} + for _, row in ipairs(rows) do views[#views + 1] = row_view(row) end + hydrate(views) + return views +end + +local function get_location() + keys_only(params, { uri = true }) + local uri = scalar(params, "uri") + if not uri or not valid_uri(uri) then invalid("uri must be a full app.certified.location AT-URI with a DID authority") end + local views = query_location(uri) + if #views == 0 then error("RecordNotFound: location record is not indexed", 0) end + return { location = views[1] } +end + +function handle() + return get_location() +end diff --git a/hypercerts-api/lua/endpoints/listLocations.lua b/hypercerts-api/lua/endpoints/listLocations.lua new file mode 100644 index 00000000..04d769f2 --- /dev/null +++ b/hypercerts-api/lua/endpoints/listLocations.lua @@ -0,0 +1,214 @@ +local COLLECTION = "app.certified.location" +local PROFILE = "app.certified.actor.profile" +local ORGANIZATION = "app.certified.actor.organization" +local NULL = json.decode("null") + +local function invalid(message) + error("InvalidRequest: " .. message, 0) +end + +local function keys_only(values, allowed) + for key in pairs(values) do + if not allowed[key] then invalid("unknown query parameter") end + end +end + +local function scalar(params, key) + local value = params[key] + if value == nil then return nil end + if type(value) ~= "string" and type(value) ~= "number" then + invalid(key .. " must occur once") + end + return tostring(value) +end + +local function valid_did(value) + if #value > 2048 then return false end + local method, specific = value:match("^did:([a-z]+):(.+)$") + if not method or not specific or specific:sub(-1) == ":" or specific:sub(-1) == "%" + or value:find("[^%w%.:_%%%-]") then return false end + return true +end + +local function valid_record_key(value) + return #value >= 1 and #value <= 512 and value ~= "." and value ~= ".." + and not value:find("[^%w_~%.:%-]") +end + +local function valid_uri(value) + if value:find("[?#]") then return false end + local authority, collection, rkey = value:match("^at://([^/]+)/([^/]+)/([^/]+)$") + return authority ~= nil and valid_did(authority) and collection == COLLECTION and valid_record_key(rkey) +end + +local function query(sql, values) + local ok, result = pcall(db.raw, sql, values) + if not ok then error("LocationQueryFailed: location lookup failed", 0) end + return result +end + +local function row_view(row) + local record = json.decode(row.record) + return { + uri = row.uri, cid = row.cid, indexedAt = row.indexed_at, did = row.did, + record = record, + } +end + +local function hydrate(views) + if #views == 0 then return end + local dids, seen = {}, {} + for _, view in ipairs(views) do + if not seen[view.did] then seen[view.did] = true; dids[#dids + 1] = view.did end + end + local profiles, organizations = {}, {} + local function load(collection, target) + if #dids == 0 then return end + local params, marks = {}, {} + params[1] = collection + for _, did in ipairs(dids) do params[#params + 1] = did; marks[#marks + 1] = "$" .. #params end + local rows = query("SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record FROM happyview_records WHERE collection = $1 AND rkey = 'self' AND did IN (" .. table.concat(marks, ",") .. ")", params) + for _, row in ipairs(rows) do target[row.did] = row end + end + load(PROFILE, profiles) + load(ORGANIZATION, organizations) + for _, view in ipairs(views) do + local profile, organization = profiles[view.did], organizations[view.did] + local author = { did = view.did, profile = NULL, organization = NULL } + if profile then author.profile = row_view(profile) end + if organization then author.organization = row_view(organization) end + view.author = author + end +end + +local function valid_datetime(value) + local year, month, day, hour, minute, second, suffix = value:match( + "^(%d%d%d%d)%-(%d%d)%-(%d%d)T(%d%d):(%d%d):(%d%d)(.*)$") + if not year then return false end + year, month, day = tonumber(year), tonumber(month), tonumber(day) + hour, minute, second = tonumber(hour), tonumber(minute), tonumber(second) + if month < 1 or month > 12 or hour > 23 or minute > 59 or second > 59 then return false end + local leap = year % 4 == 0 and (year % 100 ~= 0 or year % 400 == 0) + local month_days = { 31, leap and 29 or 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 } + if day < 1 or day > month_days[month] then return false end + local fraction, zone = suffix:match("^(%.%d+)(Z)$") + if not fraction then fraction, zone = suffix:match("^(%.%d+)([+-]%d%d:%d%d)$") end + if not fraction then zone = suffix:match("^(Z)$") end + if not zone then zone = suffix:match("^([+-]%d%d:%d%d)$") end + if not zone or zone == "-00:00" then return false end + if zone ~= "Z" then + local zh, zm = zone:match("^[+-](%d%d):(%d%d)$") + if not zh or tonumber(zh) > 23 or tonumber(zm) > 59 then return false end + end + return true +end + +local function array(params, key, validate, description, max_bytes) + local value = params[key] + if value == nil then return nil end + local values = {} + if type(value) == "string" then values[1] = value + elseif type(value) == "table" then + for i = 1, #value do + if type(value[i]) ~= "string" then invalid(key .. " entries must be strings") end + values[#values + 1] = value[i] + end + else invalid(key .. " must be a string or repeated string parameter") end + if #values > 100 then invalid(key .. " accepts at most 100 values") end + local unique, seen = {}, {} + for _, item in ipairs(values) do + if max_bytes and #item > max_bytes then invalid(key .. " entries must be at most " .. max_bytes .. " UTF-8 bytes") end + if validate and not validate(item) then invalid("each " .. key .. " value must be " .. description) end + if not seen[item] then seen[item] = true; unique[#unique + 1] = item end + end + return unique +end + +local function add_in(where, params, column, values) + if not values then return end + if #values == 0 then where[#where + 1] = "FALSE"; return end + local placeholders = {} + for _, value in ipairs(values) do + params[#params + 1] = value + placeholders[#placeholders + 1] = "$" .. #params + end + where[#where + 1] = column .. " IN (" .. table.concat(placeholders, ",") .. ")" +end + +local function cursor_encode(value) + local encoded = json.encode(value) + return (encoded:gsub(".", function(char) return string.format("%02x", string.byte(char)) end)) +end + +local function cursor_decode(token, direction) + if not token then return nil end + if #token % 2 ~= 0 or token:find("[^0-9a-f]") then invalid("cursor is malformed") end + local decoded = token:gsub("..", function(pair) return string.char(tonumber(pair, 16)) end) + local ok, value = pcall(json.decode, decoded) + if not ok or type(value) ~= "table" or value.v ~= 1 or value.d ~= direction + or type(value.t) ~= "string" or type(value.u) ~= "string" then + invalid("cursor is malformed or belongs to another sortDirection") + end + for key in pairs(value) do + if key ~= "v" and key ~= "d" and key ~= "t" and key ~= "u" then invalid("cursor is malformed") end + end + if not valid_uri(value.u) or not valid_datetime(value.t) then invalid("cursor is malformed") end + return value +end + +local function query_locations(filters, limit, cursor, direction) + if db.backend() ~= "postgres" then error("LocationQueryFailed: location API requires PostgreSQL", 0) end + local where, binds = { "collection = $1" }, { COLLECTION } + add_in(where, binds, "did", filters.authors) + add_in(where, binds, "uri", filters.uris) + add_in(where, binds, "(record::jsonb)->>'locationType'", filters.locationTypes) + if cursor then + binds[#binds + 1] = cursor.t + local time = "$" .. #binds + binds[#binds + 1] = cursor.u + local uri = "$" .. #binds + local op = direction == "asc" and ">" or "<" + where[#where + 1] = "(sorted.sort_at, uri) " .. op .. " ((" .. time .. ")::timestamptz, " .. uri .. ")" + end + binds[#binds + 1] = limit + 1 + local ordering = direction == "asc" and "ASC" or "DESC" + -- CASE guards the cast itself; a WHERE regex cannot protect it from planner reordering. + local created = "record::jsonb->>'createdAt'" + local zoned = "^[0-9]{4}-[0-9]{2}-[0-9]{2}T([01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]([.][0-9]+)?(Z|[+-]([01][0-9]|2[0-3]):[0-5][0-9])$" + local sort_key = "CASE WHEN jsonb_typeof(record::jsonb->'createdAt') = 'string' AND " .. created .. " ~ '" .. zoned .. "' AND " .. created .. " !~ '-00:00$' AND pg_input_is_valid(" .. created .. ", 'timestamptz') THEN (" .. created .. ")::timestamptz ELSE COALESCE(indexed_at::timestamptz, created_at::timestamptz) END" + local sql = "SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record, to_char(sorted.sort_at AT TIME ZONE 'UTC', 'YYYY-MM-DD\"T\"HH24:MI:SS.US\"Z\"') AS sort_timestamp FROM happyview_records CROSS JOIN LATERAL (SELECT " .. sort_key .. " AS sort_at) sorted WHERE " .. table.concat(where, " AND ") .. " ORDER BY sorted.sort_at " .. ordering .. ", uri " .. ordering .. " LIMIT $" .. #binds + local rows = query(sql, binds) + local more = #rows > limit + if more then rows[#rows] = nil end + local views = {} + for _, row in ipairs(rows) do views[#views + 1] = row_view(row) end + hydrate(views) + local next_cursor + if more then + local last = rows[#rows] + next_cursor = cursor_encode({ v = 1, d = direction, t = last.sort_timestamp, u = last.uri }) + end + return views, next_cursor +end + +local function list_locations() + keys_only(params, { authors = true, uris = true, locationTypes = true, limit = true, cursor = true, sortDirection = true }) + local authors = array(params, "authors", valid_did, "valid DIDs") + local uris = array(params, "uris", valid_uri, "full app.certified.location AT-URIs with DID authorities") + local types = array(params, "locationTypes", nil, nil, 20) + local limit_value = scalar(params, "limit") + if limit_value and not limit_value:match("^%d+$") then invalid("limit must be an integer from 1 through 100") end + local limit = limit_value and tonumber(limit_value) or 25 + if not limit or limit % 1 ~= 0 or limit < 1 or limit > 100 then invalid("limit must be an integer from 1 through 100") end + local direction = scalar(params, "sortDirection") or "desc" + if direction ~= "asc" and direction ~= "desc" then invalid("sortDirection must be 'asc' or 'desc'") end + local cursor = cursor_decode(scalar(params, "cursor"), direction) + local views, next_cursor = query_locations({ authors = authors, uris = uris, locationTypes = types }, limit, cursor, direction) + local response = { locations = toarray(views) } + if next_cursor then response.cursor = next_cursor end + return response +end + +function handle() + return list_locations() +end diff --git a/hypercerts-api/lua/shared/location.lua b/hypercerts-api/lua/shared/location.lua new file mode 100644 index 00000000..8c8037c5 --- /dev/null +++ b/hypercerts-api/lua/shared/location.lua @@ -0,0 +1,82 @@ +local COLLECTION = "app.certified.location" +local PROFILE = "app.certified.actor.profile" +local ORGANIZATION = "app.certified.actor.organization" +local NULL = json.decode("null") + +local function invalid(message) + error("InvalidRequest: " .. message, 0) +end + +local function keys_only(values, allowed) + for key in pairs(values) do + if not allowed[key] then invalid("unknown query parameter") end + end +end + +local function scalar(params, key) + local value = params[key] + if value == nil then return nil end + if type(value) ~= "string" and type(value) ~= "number" then + invalid(key .. " must occur once") + end + return tostring(value) +end + +local function valid_did(value) + if #value > 2048 then return false end + local method, specific = value:match("^did:([a-z]+):(.+)$") + if not method or not specific or specific:sub(-1) == ":" or specific:sub(-1) == "%" + or value:find("[^%w%.:_%%%-]") then return false end + return true +end + +local function valid_record_key(value) + return #value >= 1 and #value <= 512 and value ~= "." and value ~= ".." + and not value:find("[^%w_~%.:%-]") +end + +local function valid_uri(value) + if value:find("[?#]") then return false end + local authority, collection, rkey = value:match("^at://([^/]+)/([^/]+)/([^/]+)$") + return authority ~= nil and valid_did(authority) and collection == COLLECTION and valid_record_key(rkey) +end + +local function query(sql, values) + local ok, result = pcall(db.raw, sql, values) + if not ok then error("LocationQueryFailed: location lookup failed", 0) end + return result +end + +local function row_view(row) + local record = json.decode(row.record) + return { + uri = row.uri, cid = row.cid, indexedAt = row.indexed_at, did = row.did, + record = record, + } +end + +local function hydrate(views) + if #views == 0 then return end + local dids, seen = {}, {} + for _, view in ipairs(views) do + if not seen[view.did] then seen[view.did] = true; dids[#dids + 1] = view.did end + end + local profiles, organizations = {}, {} + local function load(collection, target) + if #dids == 0 then return end + local params, marks = {}, {} + params[1] = collection + for _, did in ipairs(dids) do params[#params + 1] = did; marks[#marks + 1] = "$" .. #params end + local rows = query("SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record FROM happyview_records WHERE collection = $1 AND rkey = 'self' AND did IN (" .. table.concat(marks, ",") .. ")", params) + for _, row in ipairs(rows) do target[row.did] = row end + end + load(PROFILE, profiles) + load(ORGANIZATION, organizations) + for _, view in ipairs(views) do + local profile, organization = profiles[view.did], organizations[view.did] + local author = { did = view.did, profile = NULL, organization = NULL } + if profile then author.profile = row_view(profile) end + if organization then author.organization = row_view(organization) end + view.author = author + end +end diff --git a/hypercerts-api/lua/src/getLocation.lua b/hypercerts-api/lua/src/getLocation.lua new file mode 100644 index 00000000..578bb781 --- /dev/null +++ b/hypercerts-api/lua/src/getLocation.lua @@ -0,0 +1,21 @@ +local function query_location(uri) + if db.backend() ~= "postgres" then error("LocationQueryFailed: location API requires PostgreSQL", 0) end + local rows = query("SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record FROM happyview_records WHERE collection = $1 AND uri = $2 LIMIT 1", { COLLECTION, uri }) + local views = {} + for _, row in ipairs(rows) do views[#views + 1] = row_view(row) end + hydrate(views) + return views +end + +local function get_location() + keys_only(params, { uri = true }) + local uri = scalar(params, "uri") + if not uri or not valid_uri(uri) then invalid("uri must be a full app.certified.location AT-URI with a DID authority") end + local views = query_location(uri) + if #views == 0 then error("RecordNotFound: location record is not indexed", 0) end + return { location = views[1] } +end + +function handle() + return get_location() +end diff --git a/hypercerts-api/lua/src/listLocations.lua b/hypercerts-api/lua/src/listLocations.lua new file mode 100644 index 00000000..45b178cd --- /dev/null +++ b/hypercerts-api/lua/src/listLocations.lua @@ -0,0 +1,131 @@ +local function valid_datetime(value) + local year, month, day, hour, minute, second, suffix = value:match( + "^(%d%d%d%d)%-(%d%d)%-(%d%d)T(%d%d):(%d%d):(%d%d)(.*)$") + if not year then return false end + year, month, day = tonumber(year), tonumber(month), tonumber(day) + hour, minute, second = tonumber(hour), tonumber(minute), tonumber(second) + if month < 1 or month > 12 or hour > 23 or minute > 59 or second > 59 then return false end + local leap = year % 4 == 0 and (year % 100 ~= 0 or year % 400 == 0) + local month_days = { 31, leap and 29 or 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 } + if day < 1 or day > month_days[month] then return false end + local fraction, zone = suffix:match("^(%.%d+)(Z)$") + if not fraction then fraction, zone = suffix:match("^(%.%d+)([+-]%d%d:%d%d)$") end + if not fraction then zone = suffix:match("^(Z)$") end + if not zone then zone = suffix:match("^([+-]%d%d:%d%d)$") end + if not zone or zone == "-00:00" then return false end + if zone ~= "Z" then + local zh, zm = zone:match("^[+-](%d%d):(%d%d)$") + if not zh or tonumber(zh) > 23 or tonumber(zm) > 59 then return false end + end + return true +end + +local function array(params, key, validate, description, max_bytes) + local value = params[key] + if value == nil then return nil end + local values = {} + if type(value) == "string" then values[1] = value + elseif type(value) == "table" then + for i = 1, #value do + if type(value[i]) ~= "string" then invalid(key .. " entries must be strings") end + values[#values + 1] = value[i] + end + else invalid(key .. " must be a string or repeated string parameter") end + if #values > 100 then invalid(key .. " accepts at most 100 values") end + local unique, seen = {}, {} + for _, item in ipairs(values) do + if max_bytes and #item > max_bytes then invalid(key .. " entries must be at most " .. max_bytes .. " UTF-8 bytes") end + if validate and not validate(item) then invalid("each " .. key .. " value must be " .. description) end + if not seen[item] then seen[item] = true; unique[#unique + 1] = item end + end + return unique +end + +local function add_in(where, params, column, values) + if not values then return end + if #values == 0 then where[#where + 1] = "FALSE"; return end + local placeholders = {} + for _, value in ipairs(values) do + params[#params + 1] = value + placeholders[#placeholders + 1] = "$" .. #params + end + where[#where + 1] = column .. " IN (" .. table.concat(placeholders, ",") .. ")" +end + +local function cursor_encode(value) + local encoded = json.encode(value) + return (encoded:gsub(".", function(char) return string.format("%02x", string.byte(char)) end)) +end + +local function cursor_decode(token, direction) + if not token then return nil end + if #token % 2 ~= 0 or token:find("[^0-9a-f]") then invalid("cursor is malformed") end + local decoded = token:gsub("..", function(pair) return string.char(tonumber(pair, 16)) end) + local ok, value = pcall(json.decode, decoded) + if not ok or type(value) ~= "table" or value.v ~= 1 or value.d ~= direction + or type(value.t) ~= "string" or type(value.u) ~= "string" then + invalid("cursor is malformed or belongs to another sortDirection") + end + for key in pairs(value) do + if key ~= "v" and key ~= "d" and key ~= "t" and key ~= "u" then invalid("cursor is malformed") end + end + if not valid_uri(value.u) or not valid_datetime(value.t) then invalid("cursor is malformed") end + return value +end + +local function query_locations(filters, limit, cursor, direction) + if db.backend() ~= "postgres" then error("LocationQueryFailed: location API requires PostgreSQL", 0) end + local where, binds = { "collection = $1" }, { COLLECTION } + add_in(where, binds, "did", filters.authors) + add_in(where, binds, "uri", filters.uris) + add_in(where, binds, "(record::jsonb)->>'locationType'", filters.locationTypes) + if cursor then + binds[#binds + 1] = cursor.t + local time = "$" .. #binds + binds[#binds + 1] = cursor.u + local uri = "$" .. #binds + local op = direction == "asc" and ">" or "<" + where[#where + 1] = "(sorted.sort_at, uri) " .. op .. " ((" .. time .. ")::timestamptz, " .. uri .. ")" + end + binds[#binds + 1] = limit + 1 + local ordering = direction == "asc" and "ASC" or "DESC" + -- CASE guards the cast itself; a WHERE regex cannot protect it from planner reordering. + local created = "record::jsonb->>'createdAt'" + local zoned = "^[0-9]{4}-[0-9]{2}-[0-9]{2}T([01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]([.][0-9]+)?(Z|[+-]([01][0-9]|2[0-3]):[0-5][0-9])$" + local sort_key = "CASE WHEN jsonb_typeof(record::jsonb->'createdAt') = 'string' AND " .. created .. " ~ '" .. zoned .. "' AND " .. created .. " !~ '-00:00$' AND pg_input_is_valid(" .. created .. ", 'timestamptz') THEN (" .. created .. ")::timestamptz ELSE COALESCE(indexed_at::timestamptz, created_at::timestamptz) END" + local sql = "SELECT uri, did, cid, indexed_at::text AS indexed_at, record::text AS record, to_char(sorted.sort_at AT TIME ZONE 'UTC', 'YYYY-MM-DD\"T\"HH24:MI:SS.US\"Z\"') AS sort_timestamp FROM happyview_records CROSS JOIN LATERAL (SELECT " .. sort_key .. " AS sort_at) sorted WHERE " .. table.concat(where, " AND ") .. " ORDER BY sorted.sort_at " .. ordering .. ", uri " .. ordering .. " LIMIT $" .. #binds + local rows = query(sql, binds) + local more = #rows > limit + if more then rows[#rows] = nil end + local views = {} + for _, row in ipairs(rows) do views[#views + 1] = row_view(row) end + hydrate(views) + local next_cursor + if more then + local last = rows[#rows] + next_cursor = cursor_encode({ v = 1, d = direction, t = last.sort_timestamp, u = last.uri }) + end + return views, next_cursor +end + +local function list_locations() + keys_only(params, { authors = true, uris = true, locationTypes = true, limit = true, cursor = true, sortDirection = true }) + local authors = array(params, "authors", valid_did, "valid DIDs") + local uris = array(params, "uris", valid_uri, "full app.certified.location AT-URIs with DID authorities") + local types = array(params, "locationTypes", nil, nil, 20) + local limit_value = scalar(params, "limit") + if limit_value and not limit_value:match("^%d+$") then invalid("limit must be an integer from 1 through 100") end + local limit = limit_value and tonumber(limit_value) or 25 + if not limit or limit % 1 ~= 0 or limit < 1 or limit > 100 then invalid("limit must be an integer from 1 through 100") end + local direction = scalar(params, "sortDirection") or "desc" + if direction ~= "asc" and direction ~= "desc" then invalid("sortDirection must be 'asc' or 'desc'") end + local cursor = cursor_decode(scalar(params, "cursor"), direction) + local views, next_cursor = query_locations({ authors = authors, uris = uris, locationTypes = types }, limit, cursor, direction) + local response = { locations = toarray(views) } + if next_cursor then response.cursor = next_cursor end + return response +end + +function handle() + return list_locations() +end diff --git a/hypercerts-api/manifest.json b/hypercerts-api/manifest.json new file mode 100644 index 00000000..3fa86f90 --- /dev/null +++ b/hypercerts-api/manifest.json @@ -0,0 +1,167 @@ +{ + "name": "hypercerts-location-api", + "collection": "app.certified.location", + "targetHappyViewRevision": "a2f347e605d7cd9c11669b320e43c10dbfe982e3", + "runtimeCompatibility": "pending-runtime-validation", + "status": "handlers-implemented-runtime-validation-pending", + "handlerStatus": { + "getLocation": "implemented", + "listLocations": "implemented" + }, + "authentication": { + "decision": "public", + "unresolved": false, + "note": "Both endpoints expose public record reads and require no caller authentication." + }, + "luaBuild": { + "command": "pnpm build:lua", + "sharedSource": "lua/shared/location.lua", + "endpointSources": ["lua/src/getLocation.lua", "lua/src/listLocations.lua"], + "installedHandlers": "lua/endpoints/" + }, + "validationLexicons": [ + { + "id": "app.certified.location", + "packagePath": "lexicons/app/certified/location.json" + }, + { + "id": "org.hypercerts.api.defs", + "path": "lexicons/org.hypercerts.api.defs.json" + }, + { + "id": "app.certified.location.getLocation", + "path": "lexicons/app.certified.location.getLocation.json" + }, + { + "id": "app.certified.location.listLocations", + "path": "lexicons/app.certified.location.listLocations.json" + }, + { + "id": "app.certified.actor.profile", + "packagePath": "lexicons/app/certified/actor/profile.json" + }, + { + "id": "app.certified.actor.organization", + "packagePath": "lexicons/app/certified/actor/organization.json" + }, + { + "id": "app.bsky.richtext.facet", + "packagePath": "lexicons/app/bsky/richtext/facet.json" + }, + { + "id": "app.certified.signature.defs", + "packagePath": "lexicons/app/certified/signature/defs.json" + }, + { + "id": "com.atproto.repo.strongRef", + "packagePath": "lexicons/com/atproto/repo/strongRef.json" + }, + { + "id": "org.hypercerts.defs", + "packagePath": "lexicons/org/hypercerts/defs.json" + }, + { + "id": "pub.leaflet.blocks.blockquote", + "packagePath": "lexicons/pub/leaflet/blocks/blockquote.json" + }, + { + "id": "pub.leaflet.blocks.bskyPost", + "packagePath": "lexicons/pub/leaflet/blocks/bskyPost.json" + }, + { + "id": "pub.leaflet.blocks.button", + "packagePath": "lexicons/pub/leaflet/blocks/button.json" + }, + { + "id": "pub.leaflet.blocks.code", + "packagePath": "lexicons/pub/leaflet/blocks/code.json" + }, + { + "id": "pub.leaflet.blocks.header", + "packagePath": "lexicons/pub/leaflet/blocks/header.json" + }, + { + "id": "pub.leaflet.blocks.horizontalRule", + "packagePath": "lexicons/pub/leaflet/blocks/horizontalRule.json" + }, + { + "id": "pub.leaflet.blocks.html", + "packagePath": "lexicons/pub/leaflet/blocks/html.json" + }, + { + "id": "pub.leaflet.blocks.iframe", + "packagePath": "lexicons/pub/leaflet/blocks/iframe.json" + }, + { + "id": "pub.leaflet.blocks.image", + "packagePath": "lexicons/pub/leaflet/blocks/image.json" + }, + { + "id": "pub.leaflet.blocks.imageGallery", + "packagePath": "lexicons/pub/leaflet/blocks/imageGallery.json" + }, + { + "id": "pub.leaflet.blocks.math", + "packagePath": "lexicons/pub/leaflet/blocks/math.json" + }, + { + "id": "pub.leaflet.blocks.membersOnlyDelimiter", + "packagePath": "lexicons/pub/leaflet/blocks/membersOnlyDelimiter.json" + }, + { + "id": "pub.leaflet.blocks.orderedList", + "packagePath": "lexicons/pub/leaflet/blocks/orderedList.json" + }, + { + "id": "pub.leaflet.blocks.page", + "packagePath": "lexicons/pub/leaflet/blocks/page.json" + }, + { + "id": "pub.leaflet.blocks.poll", + "packagePath": "lexicons/pub/leaflet/blocks/poll.json" + }, + { + "id": "pub.leaflet.blocks.postsList", + "packagePath": "lexicons/pub/leaflet/blocks/postsList.json" + }, + { + "id": "pub.leaflet.blocks.signup", + "packagePath": "lexicons/pub/leaflet/blocks/signup.json" + }, + { + "id": "pub.leaflet.blocks.standardSitePost", + "packagePath": "lexicons/pub/leaflet/blocks/standardSitePost.json" + }, + { + "id": "pub.leaflet.blocks.standardSitePublication", + "packagePath": "lexicons/pub/leaflet/blocks/standardSitePublication.json" + }, + { + "id": "pub.leaflet.blocks.text", + "packagePath": "lexicons/pub/leaflet/blocks/text.json" + }, + { + "id": "pub.leaflet.blocks.unorderedList", + "packagePath": "lexicons/pub/leaflet/blocks/unorderedList.json" + }, + { + "id": "pub.leaflet.blocks.website", + "packagePath": "lexicons/pub/leaflet/blocks/website.json" + }, + { + "id": "pub.leaflet.pages.linearDocument", + "packagePath": "lexicons/pub/leaflet/pages/linearDocument.json" + }, + { + "id": "pub.leaflet.richtext.facet", + "packagePath": "lexicons/pub/leaflet/richtext/facet.json" + }, + { + "id": "pub.leaflet.theme.color", + "packagePath": "lexicons/pub/leaflet/theme/color.json" + } + ], + "modules": [ + "modules/location/manifest.json" + ] +} diff --git a/hypercerts-api/modules/location/manifest.json b/hypercerts-api/modules/location/manifest.json new file mode 100644 index 00000000..d5f3fd9e --- /dev/null +++ b/hypercerts-api/modules/location/manifest.json @@ -0,0 +1,119 @@ +{ + "assets": [ + { + "kind": "lexicon", + "id": "app.certified.actor.organization", + "config": { + "backfill": true + }, + "dependsOn": [], + "registrationGroup": "location-record-schema-sources", + "packagePath": "lexicons/app/certified/actor/organization.json" + }, + { + "kind": "lexicon", + "id": "app.certified.actor.profile", + "config": { + "backfill": true + }, + "dependsOn": [], + "registrationGroup": "location-record-schema-sources", + "packagePath": "lexicons/app/certified/actor/profile.json" + }, + { + "kind": "lexicon", + "id": "app.certified.location", + "config": { + "backfill": true + }, + "dependsOn": [], + "registrationGroup": "location-record-schema-sources", + "packagePath": "lexicons/app/certified/location.json" + }, + { + "kind": "lexicon", + "id": "app.certified.signature.defs", + "config": { + "backfill": false + }, + "dependsOn": [], + "registrationGroup": "location-record-schema-sources", + "packagePath": "lexicons/app/certified/signature/defs.json" + }, + { + "kind": "lexicon", + "id": "org.hypercerts.defs", + "config": { + "backfill": false + }, + "dependsOn": [], + "registrationGroup": "location-record-schema-sources", + "packagePath": "lexicons/org/hypercerts/defs.json" + }, + { + "kind": "lexicon", + "id": "org.hypercerts.api.defs", + "path": "../../lexicons/org.hypercerts.api.defs.json", + "config": { + "backfill": false + }, + "dependsOn": [ + "app.certified.actor.organization", + "app.certified.actor.profile" + ] + }, + { + "kind": "lexicon", + "id": "app.certified.location.getLocation", + "path": "../../lexicons/app.certified.location.getLocation.json", + "config": { + "backfill": false, + "target_collection": "app.certified.location" + }, + "dependsOn": [ + "app.certified.location", + "org.hypercerts.api.defs" + ] + }, + { + "kind": "lexicon", + "id": "app.certified.location.listLocations", + "path": "../../lexicons/app.certified.location.listLocations.json", + "config": { + "backfill": false, + "target_collection": "app.certified.location" + }, + "dependsOn": [ + "app.certified.location.getLocation" + ] + }, + { + "kind": "script", + "id": "xrpc.query:app.certified.location.getLocation", + "path": "../../lua/endpoints/getLocation.lua", + "sourcePath": "../../lua/src/getLocation.lua", + "sharedSourcePath": "../../lua/shared/location.lua", + "config": { + "script_type": "lua", + "description": "Location API handler" + }, + "dependsOn": [ + "app.certified.location.getLocation" + ] + }, + { + "kind": "script", + "id": "xrpc.query:app.certified.location.listLocations", + "path": "../../lua/endpoints/listLocations.lua", + "sourcePath": "../../lua/src/listLocations.lua", + "sharedSourcePath": "../../lua/shared/location.lua", + "config": { + "script_type": "lua", + "description": "Location API handler" + }, + "dependsOn": [ + "app.certified.location.listLocations" + ] + } + ] +} diff --git a/hypercerts-api/package.json b/hypercerts-api/package.json index 7b2b3f70..f4f7a8e1 100644 --- a/hypercerts-api/package.json +++ b/hypercerts-api/package.json @@ -15,7 +15,11 @@ "lint": "pnpm run lint:js && pnpm run lint:lua", "typecheck": "tsc --project tsconfig.json", "check": "pnpm run lint && pnpm run typecheck && pnpm run test:unit", - "test:unit": "node --test tooling/*.test.js tests/fixtures/*.test.js tests/contracts/helpers.test.js", + "build:lua": "node tooling/build-lua.js", + "test:unit": "node --test tooling/*.test.js tests/fixtures/*.test.js tests/contracts/helpers.test.js tests/contracts/location-filters.test.js", + "test:contracts": "node --test tests/contracts/location.contract.test.js", + "test:bad-dates": "node --test tests/contracts/location-bad-dates.contract.test.js", + "install:api": "node tooling/installer.js", "seed:test": "node tooling/seed.js", "seed:bad-dates": "node tooling/seed.js --bad-dates" }, diff --git a/hypercerts-api/tests/contracts/location-bad-dates.contract.test.js b/hypercerts-api/tests/contracts/location-bad-dates.contract.test.js new file mode 100644 index 00000000..ffe1f1b1 --- /dev/null +++ b/hypercerts-api/tests/contracts/location-bad-dates.contract.test.js @@ -0,0 +1,54 @@ +// Run only after badDateSeedSql has been loaded into an approved disposable target. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { badDateLocations } from '../fixtures/bad-location-dates.js'; +import { contractUrl, requireContractTarget } from './helpers.js'; + +const baseUrl = requireContractTarget(); +const byKey = new Map(badDateLocations.map((row) => [row.rkey, row])); +const asc = ['goodearlier', 'badarray', 'badcalendar', 'badhour', 'badmissing', 'badnozone', 'badnull', 'badnumber', 'badtext', 'badunindexed', 'badzone', 'goodfraction', 'goodoffset', 'goodnano']; +const tiedAt = '2025-01-02T03:04:05.123456Z'; +const expectedTime = (rkey) => rkey === 'goodearlier' ? '2025-01-01T00:00:00.000000Z' : rkey === 'goodnano' ? '2025-01-02T03:04:05.123457Z' : tiedAt; + +async function request(nsid, params) { + const response = await fetch(contractUrl(baseUrl, nsid, params)); + const body = await response.json(); + assert.equal(response.status, 200, `${nsid}: ${JSON.stringify(body)}`); + return body; +} + +test('bad or absent createdAt falls back without hiding or changing records in getLocation', async () => { + for (const fixture of badDateLocations) { + const body = await request('app.certified.location.getLocation', { uri: fixture.uri }); + assert.equal(body.location.uri, fixture.uri); + assert.deepEqual(body.location.record, fixture.record); + } +}); + +test('bad dates and equal instants paginate without gaps in either direction using the DB key', async () => { + const uris = badDateLocations.map(({ uri }) => uri); + for (const direction of ['asc', 'desc']) { + const expected = direction === 'asc' ? asc : [...asc].reverse(); + const seen = []; + let cursor; + do { + const page = await request('app.certified.location.listLocations', { + uris, limit: 3, sortDirection: direction, cursor, + }); + for (const view of page.locations) { + const key = view.uri.slice(view.uri.lastIndexOf('/') + 1); + seen.push(key); + assert.deepEqual(view.record, byKey.get(key).record); + } + cursor = page.cursor; + if (cursor) { + const token = JSON.parse(Buffer.from(cursor, 'hex').toString('utf8')); + const last = seen.at(-1); + assert.equal(token.t, expectedTime(last)); + assert.equal(token.u, byKey.get(last).uri); + assert.equal(token.d, direction); + } + } while (cursor); + assert.deepEqual(seen, expected); + } +}); diff --git a/hypercerts-api/tests/contracts/location-filters.test.js b/hypercerts-api/tests/contracts/location-filters.test.js new file mode 100644 index 00000000..9a3e5d5e --- /dev/null +++ b/hypercerts-api/tests/contracts/location-filters.test.js @@ -0,0 +1,12 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { contractUrl } from './helpers.js'; + +test('listLocations filter names and values serialize as expected', () => { + assert.equal(contractUrl('http://127.0.0.1:8080', 'app.certified.location.listLocations', { + authors: ['did:plc:a', 'did:plc:b'], + uris: ['at://did:plc:a/app.certified.location/3jzfcijpj2z2a'], + locationTypes: ['geojson', 'address'], + sortDirection: 'asc', + }), 'http://127.0.0.1:8080/xrpc/app.certified.location.listLocations?authors=did%3Aplc%3Aa&authors=did%3Aplc%3Ab&uris=at%3A%2F%2Fdid%3Aplc%3Aa%2Fapp.certified.location%2F3jzfcijpj2z2a&locationTypes=geojson&locationTypes=address&sortDirection=asc'); +}); diff --git a/hypercerts-api/tests/contracts/location.contract.test.js b/hypercerts-api/tests/contracts/location.contract.test.js new file mode 100644 index 00000000..dc380ff6 --- /dev/null +++ b/hypercerts-api/tests/contracts/location.contract.test.js @@ -0,0 +1,158 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { locationRecords, profileRecords, organizationRecords } from '../fixtures/records.js'; +import { contractUrl, requireContractTarget } from './helpers.js'; + +const baseUrl = requireContractTarget(); +const [forest, river, blobLocation, emptyTypeLocation, noRelations, profileOnly, organizationOnly] = locationRecords; + +async function get(nsid, params) { + const response = await fetch(contractUrl(baseUrl, nsid, params)); + const body = await response.json(); + assert.equal(response.status, 200, `${nsid}: ${JSON.stringify(body)}`); + return body; +} + +function cursorWithTimestamp(cursor, timestamp) { + const value = JSON.parse(Buffer.from(cursor, 'hex').toString('utf8')); + value.t = timestamp; + return Buffer.from(JSON.stringify(value), 'utf8').toString('hex'); +} + +async function assertDomainError(response, code) { + assert.equal(response.status, 500); + const body = await response.json(); + assert.equal(body.error, 'script_error'); + assert.equal(body.errorType, 'runtime'); + assert.match(body.message, new RegExp(`^runtime error: ${code}:`)); + assert.doesNotMatch(body.message, /SELECT|happyview_records|at:\/\//); +} + +test('getLocation returns complete location metadata and nullable hydrated author relations', async () => { + const result = await get('app.certified.location.getLocation', { uri: forest.uri }); + assert.equal(result.location.uri, forest.uri); + assert.equal(result.location.cid, forest.cid); + assert.equal(result.location.did, forest.did); + assert.ok(result.location.indexedAt); + assert.equal(result.location.record.$type, 'app.certified.location'); + assert.equal(result.location.record.name, 'Thimphu Forest'); + assert.equal(result.location.author.did, forest.did); + assert.deepEqual(result.location.author.profile, { + uri: profileRecords[0].uri, cid: profileRecords[0].cid, did: profileRecords[0].did, + indexedAt: profileRecords[0].indexedAt, record: profileRecords[0].record, + }); + assert.deepEqual(result.location.author.organization, { + uri: organizationRecords[0].uri, cid: organizationRecords[0].cid, did: organizationRecords[0].did, + indexedAt: organizationRecords[0].indexedAt, record: organizationRecords[0].record, + }); +}); + +test('getLocation preserves a blob-backed location reference', async () => { + const result = await get('app.certified.location.getLocation', { uri: blobLocation.uri }); + assert.deepEqual(result.location.record.location, blobLocation.record.location); +}); + +test('getLocation missing URI fails as a record-not-found error, not an empty view', async () => { + const response = await fetch(contractUrl(baseUrl, 'app.certified.location.getLocation', { uri: 'at://did:plc:abcdefghijklmnopqrstuvwx/app.certified.location/3jzfcijpj2z2z' })); + await assertDomainError(response, 'RecordNotFound'); +}); + +test('listLocations applies repeated authors, URI and open location-type filters together', async () => { + const result = await get('app.certified.location.listLocations', { + authors: [forest.did], uris: [forest.uri, river.uri], locationTypes: ['geojson', 'address'], limit: 100, sortDirection: 'asc', + }); + assert.deepEqual(result.locations.map(({ uri }) => uri).sort(), [forest.uri, river.uri].sort()); + assert.equal(result.cursor, undefined); +}); + +test('listLocations returns an empty array and omits a terminal cursor', async () => { + const result = await get('app.certified.location.listLocations', { locationTypes: ['not-a-fixture-type'] }); + assert.deepEqual(result.locations, []); + assert.equal(result.cursor, undefined); +}); + +test('author hydration keeps records visible for all profile and organization combinations', async () => { + const result = await get('app.certified.location.listLocations', { uris: [noRelations.uri, profileOnly.uri, organizationOnly.uri] }); + const byUri = new Map(result.locations.map((location) => [location.uri, location.author])); + assert.deepEqual(byUri.get(noRelations.uri), { did: noRelations.did, profile: null, organization: null }); + assert.equal(byUri.get(profileOnly.uri).profile.did, profileOnly.did); + assert.equal(byUri.get(profileOnly.uri).organization, null); + assert.equal(byUri.get(organizationOnly.uri).profile, null); + assert.equal(byUri.get(organizationOnly.uri).organization.did, organizationOnly.did); +}); + +test('empty locationType matches the fixture value', async () => { + const result = await get('app.certified.location.listLocations', { locationTypes: [''] }); + assert.deepEqual(result.locations.map(({ uri }) => uri), [emptyTypeLocation.uri]); +}); + +test('listLocations paginates deterministically in both directions without changing filters', async () => { + for (const direction of ['asc', 'desc']) { + const all = []; + let cursor; + do { + const page = await get('app.certified.location.listLocations', { authors: [forest.did, noRelations.did], limit: 2, sortDirection: direction, cursor }); + all.push(...page.locations.map(({ uri }) => uri)); + cursor = page.cursor; + } while (cursor); + const expected = [forest, river, blobLocation, emptyTypeLocation, noRelations].sort((a, b) => { + const delta = Date.parse(a.record.createdAt) - Date.parse(b.record.createdAt); + return (delta || a.uri.localeCompare(b.uri)) * (direction === 'asc' ? 1 : -1); + }).map(({ uri }) => uri); + assert.deepEqual(all, expected); + } +}); + +test('listLocations cursor stores the UTC database instant, not the record timezone', async () => { + const page = await get('app.certified.location.listLocations', { + uris: [forest.uri, river.uri, blobLocation.uri, emptyTypeLocation.uri, noRelations.uri], + sortDirection: 'asc', limit: 4, + }); + assert.equal(page.locations.at(-1).uri, emptyTypeLocation.uri); + const token = JSON.parse(Buffer.from(page.cursor, 'hex').toString('utf8')); + assert.equal(token.t, '2025-01-03T23:00:00.000000Z'); +}); + +test('listLocations accepts 100 supplied filter occurrences before deduplication', async () => { + const result = await get('app.certified.location.listLocations', { authors: Array(100).fill(forest.did), limit: 100 }); + assert.equal(result.locations.length, 4); + assert.equal(new Set(result.locations.map(({ uri }) => uri)).size, 4); +}); + +test('listLocations rejects malformed or direction-mismatched cursors and invalid bounds', async () => { + const firstPage = await get('app.certified.location.listLocations', { limit: 1 }); + const oppositeDirection = cursorWithTimestamp(firstPage.cursor, firstPage.locations[0].record.createdAt); + for (const params of [ + { cursor: 'bad!' }, { cursor: oppositeDirection, sortDirection: 'asc' }, + { limit: 0 }, { limit: 101 }, { authors: Array(101).fill(forest.did) }, + { authors: ['alice.example'] }, { uris: ['at://alice.example/app.certified.location/x'] }, + { search: 'forest' }, { unknown: 'value' }, { limit: [1, 2] }, + ]) { + const response = await fetch(contractUrl(baseUrl, 'app.certified.location.listLocations', params)); + await assertDomainError(response, 'InvalidRequest'); + } +}); + +test('listLocations rejects malformed cursor datetime and ATProto identifiers', async () => { + const firstPage = await get('app.certified.location.listLocations', { limit: 1 }); + assert.ok(firstPage.cursor); + const invalid = [ + ...['2025-01-01Tgarbage', '2025-02-29T00:00:00Z', '2025-13-01T00:00:00Z', '2025-01-01T24:00:00Z', '2025-01-01T00:00:60Z', '2025-01-01T00:00:00Z trailing', '2025-01-01T00:00:00', '2025-01-01T00:00:00-00:00'] + .map((timestamp) => ({ cursor: cursorWithTimestamp(firstPage.cursor, timestamp) })), + { authors: ['did:plc:abc:'] }, { authors: ['did:1:abc'] }, { authors: [`did:${'a'.repeat(2045)}`] }, + { uris: ['at://did:plc:abcdefghijklmnopqrstuvwx/app.certified.location/.'] }, + { uris: ['at://did:plc:abcdefghijklmnopqrstuvwx/app.certified.location/..'] }, + { uris: [`at://did:plc:abcdefghijklmnopqrstuvwx/app.certified.location/${'a'.repeat(513)}`] }, + ]; + for (const params of invalid) { + const response = await fetch(contractUrl(baseUrl, 'app.certified.location.listLocations', params)); + await assertDomainError(response, 'InvalidRequest'); + } + for (const timestamp of ['2024-02-29T23:59:59.123456789Z', '2025-01-01T00:00:00.123+05:30']) { + const response = await fetch(contractUrl(baseUrl, 'app.certified.location.listLocations', { + cursor: cursorWithTimestamp(firstPage.cursor, timestamp), authors: ['did:web:example.com%3A443:user'], + })); + assert.equal(response.status, 200); + assert.deepEqual((await response.json()).locations, []); + } +}); diff --git a/hypercerts-api/tests/fixtures/bad-location-dates.js b/hypercerts-api/tests/fixtures/bad-location-dates.js new file mode 100644 index 00000000..52d830b6 --- /dev/null +++ b/hypercerts-api/tests/fixtures/bad-location-dates.js @@ -0,0 +1,9 @@ +import { makeDateCaseRows } from './bad-dates.js'; +import { locationRecords } from './records.js'; + +export const badDateLocations = await makeDateCaseRows(locationRecords[0], { + did: 'did:plc:baddatefixturesexamplexx', + decorateRecord: (record, rkey) => ({ + ...record, locationType: 'date-test', name: `Date test ${rkey}`, description: 'Synthetic timestamp fixture', + }), +}); diff --git a/hypercerts-api/tooling/build-lua.js b/hypercerts-api/tooling/build-lua.js new file mode 100644 index 00000000..53b3a6eb --- /dev/null +++ b/hypercerts-api/tooling/build-lua.js @@ -0,0 +1,10 @@ +import { readFile, writeFile } from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const root = fileURLToPath(new URL('../', import.meta.url)); +const shared = await readFile(path.join(root, 'lua/shared/location.lua'), 'utf8'); +for (const name of ['getLocation', 'listLocations']) { + const endpoint = await readFile(path.join(root, `lua/src/${name}.lua`), 'utf8'); + await writeFile(path.join(root, `lua/endpoints/${name}.lua`), `${shared.trimEnd()}\n\n${endpoint}`); +} diff --git a/hypercerts-api/tooling/installer-cli.test.js b/hypercerts-api/tooling/installer-cli.test.js index 979faecf..d0fd4e81 100644 --- a/hypercerts-api/tooling/installer-cli.test.js +++ b/hypercerts-api/tooling/installer-cli.test.js @@ -5,6 +5,29 @@ import { copyFile, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; +import { resolveInstallConfig } from './installer.js'; + +test('installer prompts for missing URL and token, hiding the token input', async () => { + const prompts = []; + const answers = ['https://happyview.example', 'hv_admin-token']; + const config = await resolveInstallConfig({ + env: {}, + isTTY: true, + ask: async (label, { hidden }) => { + prompts.push({ label, hidden }); + return answers.shift(); + }, + }); + + assert.deepEqual(config, { + baseUrl: 'https://happyview.example', + token: 'hv_admin-token', + }); + assert.deepEqual(prompts, [ + { label: 'HappyView URL', hidden: false }, + { label: 'HappyView admin token', hidden: true }, + ]); +}); test('CLI requires a nonblank admin token and does not fall back to a session cookie', () => { const script = fileURLToPath(new URL('./installer.js', import.meta.url)); @@ -22,6 +45,22 @@ test('CLI requires a nonblank admin token and does not fall back to a session co } }); +test('CLI reports an actionable error for a malformed HappyView admin URL', () => { + const script = fileURLToPath(new URL('./installer.js', import.meta.url)); + const child = spawnSync(process.execPath, [script], { + encoding: 'utf8', + env: { + PATH: process.env.PATH ?? '', + HAPPYVIEW_BASE_URL: 'not a URL', + HAPPYVIEW_ADMIN_TOKEN: 'hv_cli-test-token', + }, + }); + + assert.equal(child.status, 1); + assert.equal(child.stdout, ''); + assert.equal(child.stderr, 'HappyView admin URL must be a valid HTTP(S) URL\n'); +}); + test('importing the installer does not run the CLI or require configuration', () => { const script = fileURLToPath(new URL('./installer.js', import.meta.url)); const child = spawnSync(process.execPath, ['--input-type=module', '-e', 'await import(process.argv[1])', pathToFileURL(script).href], { diff --git a/hypercerts-api/tooling/installer.js b/hypercerts-api/tooling/installer.js index 8f54c123..d6164a1e 100644 --- a/hypercerts-api/tooling/installer.js +++ b/hypercerts-api/tooling/installer.js @@ -1,6 +1,7 @@ import { readFile, stat } from 'node:fs/promises'; import { createRequire } from 'node:module'; import path from 'node:path'; +import readline from 'node:readline'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { readLexiconSource } from './lexicon-source.js'; @@ -427,19 +428,85 @@ export async function loadAssets(manifestPath) { return { assets }; } -/** @param {string} name @returns {string} */ -function requiredEnv(name) { - const value = process.env[name]; - if (typeof value !== 'string' || !value.trim()) { - throw new Error(`${name} is required and must not be blank; set it before running the installer (see hypercerts-api/README.md)`); +/** @param {string} label @param {{ hidden?: boolean }} [options] @returns {Promise} */ +function askOnTerminal(label, { hidden = false } = {}) { + return new Promise((resolve, reject) => { + const rl = readline.createInterface({ + input: process.stdin, + output: process.stdout, + terminal: true, + }); + + if (hidden) { + // Readline echoes keypresses through this hook; keep the token off stdout. + const terminal = /** @type {{ _writeToOutput: (text: string) => void }} */ (/** @type {unknown} */ (rl)); + const writeToOutput = terminal._writeToOutput.bind(rl); + let promptWritten = false; + terminal._writeToOutput = (text) => { + if (!promptWritten) { + promptWritten = true; + writeToOutput(text); + } else if (text.includes('\n') || text.includes('\r')) { + writeToOutput(text); + } + }; + } + + let settled = false; + rl.once('close', () => { + if (settled) return; + settled = true; + reject(new Error(`${label} prompt ended unexpectedly; rerun in a terminal or set its environment variable`)); + }); + rl.once('SIGINT', () => { + if (settled) return; + settled = true; + rl.close(); + reject(new Error(`${label} prompt was cancelled; rerun the installer to try again`)); + }); + rl.question(`${label}: `, (answer) => { + if (settled) return; + settled = true; + rl.close(); + resolve(answer); + }); + }); +} + +/** + * @param {{ env?: NodeJS.ProcessEnv; isTTY?: boolean; ask?: (label: string, options: { hidden?: boolean }) => Promise }} [options] + * @returns {Promise<{ baseUrl: string; token: string }>} + */ +export async function resolveInstallConfig({ + env = process.env, + isTTY = Boolean(process.stdin.isTTY && process.stdout.isTTY), + ask = askOnTerminal, +} = {}) { + /** @param {string} name @param {string} label @param {{ hidden?: boolean }} [options] @returns {Promise} */ + async function resolveValue(name, label, { hidden = false } = {}) { + const configured = typeof env[name] === 'string' ? env[name].trim() : ''; + if (configured) return configured; + + if (!isTTY) { + throw new Error(`${name} is required and must not be blank; set it in the environment or run the installer in an interactive terminal (see hypercerts-api/README.md)`); + } + + const value = await ask(label, { hidden }); + if (typeof value !== 'string' || !value.trim()) { + throw new Error(`${name} is required and must not be blank; enter a value at the prompt or set it in the environment (see hypercerts-api/README.md)`); + } + return value.trim(); } - return value.trim(); + + return { + baseUrl: await resolveValue('HAPPYVIEW_BASE_URL', 'HappyView URL'), + token: await resolveValue('HAPPYVIEW_ADMIN_TOKEN', 'HappyView admin token', { hidden: true }), + }; } async function main() { - const baseUrl = new URL(requiredEnv('HAPPYVIEW_BASE_URL')); - const token = requiredEnv('HAPPYVIEW_ADMIN_TOKEN'); - const client = createAdminClient({ baseUrl, token }); + const { baseUrl: rawBaseUrl, token } = await resolveInstallConfig(); + const client = createAdminClient({ baseUrl: rawBaseUrl, token }); const { assets } = await loadAssets(fileURLToPath(new URL('../manifest.json', import.meta.url))); const result = await applyAssets(assets, client); console.log(JSON.stringify(result, null, 2)); diff --git a/hypercerts-api/tooling/installer.test.js b/hypercerts-api/tooling/installer.test.js new file mode 100644 index 00000000..8d2988cc --- /dev/null +++ b/hypercerts-api/tooling/installer.test.js @@ -0,0 +1,62 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync, readdirSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import { loadAssets, orderAssets } from './installer.js'; + +test('installer deploys only location records, required definitions, API Lexicons, and handlers', async () => { + const manifest = JSON.parse(readFileSync(new URL('../manifest.json', import.meta.url), 'utf8')); + const { assets } = await loadAssets(fileURLToPath(new URL('../manifest.json', import.meta.url))); + const installed = new Map(assets.filter(({ kind }) => kind === 'lexicon').map((entry) => [entry.id, entry])); + const expectedAssetIds = [ + 'app.certified.actor.organization', + 'app.certified.actor.profile', + 'app.certified.location', + 'app.certified.signature.defs', + 'org.hypercerts.defs', + 'org.hypercerts.api.defs', + 'app.certified.location.getLocation', + 'app.certified.location.listLocations', + 'xrpc.query:app.certified.location.getLocation', + 'xrpc.query:app.certified.location.listLocations', + ]; + assert.deepEqual(assets.map(({ id }) => id).sort(), expectedAssetIds.sort()); + + const validationSources = new Map(manifest.validationLexicons.map((source) => [source.id, source])); + for (const asset of installed.values()) { + const isRecordLexicon = asset.lexicon_json.defs?.main?.type === 'record'; + assert.equal(asset.config.backfill, isRecordLexicon, `${asset.id} backfill configuration`); + assert.equal(asset.lexicon_json.id, asset.id); + if (asset.packagePath) assert.equal(asset.packagePath, validationSources.get(asset.id)?.packagePath); + } + const ordered = orderAssets(assets).map(({ id }) => id); + const position = (id) => ordered.indexOf(id); + const sharedViews = installed.get('org.hypercerts.api.defs'); + const locationRecord = installed.get('app.certified.location'); + const getLocation = installed.get('app.certified.location.getLocation'); + const listLocations = installed.get('app.certified.location.listLocations'); + assert.ok(sharedViews, 'missing shared API view definitions'); + assert.ok(locationRecord, 'missing pinned location record schema'); + assert.ok(getLocation, 'missing getLocation Lexicon'); + assert.ok(listLocations, 'missing listLocations Lexicon'); + assert.deepEqual(sharedViews.dependsOn, ['app.certified.actor.organization', 'app.certified.actor.profile']); + assert.deepEqual(getLocation.dependsOn, ['app.certified.location', 'org.hypercerts.api.defs']); + assert.deepEqual(listLocations.dependsOn, ['app.certified.location.getLocation']); + assert.equal(locationRecord.packagePath, 'lexicons/app/certified/location.json'); + assert.ok(position('app.certified.actor.organization') < position('org.hypercerts.api.defs')); + assert.ok(position('app.certified.actor.profile') < position('org.hypercerts.api.defs')); + assert.ok(position('app.certified.location') < position('app.certified.location.getLocation')); + assert.ok(position('org.hypercerts.api.defs') < position('app.certified.location.getLocation')); + assert.ok(position('app.certified.location.getLocation') < position('app.certified.location.listLocations')); + assert.ok(position('app.certified.location.getLocation') < position('xrpc.query:app.certified.location.getLocation')); + assert.ok(position('app.certified.location.listLocations') < position('xrpc.query:app.certified.location.listLocations')); +}); + +test('the local location API schemas remain the only checked-in schemas', () => { + const files = readdirSync(new URL('../lexicons/', import.meta.url)).filter((file) => file.endsWith('.json')); + assert.deepEqual(files.sort(), [ + 'app.certified.location.getLocation.json', + 'app.certified.location.listLocations.json', + 'org.hypercerts.api.defs.json', + ]); +}); diff --git a/hypercerts-api/tooling/lexicons.test.js b/hypercerts-api/tooling/lexicons.test.js new file mode 100644 index 00000000..8c20151c --- /dev/null +++ b/hypercerts-api/tooling/lexicons.test.js @@ -0,0 +1,97 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFile } from 'node:fs/promises'; +import { locationRecords, profileRecords, organizationRecords } from '../tests/fixtures/records.js'; +import { validatePackageLexicons } from './validate-lexicons.js'; +import { readLexiconSource } from './lexicon-source.js'; +import { loadAssets } from './installer.js'; +import { fileURLToPath } from 'node:url'; + +test('the full validation Lexicon closure resolves locally while only selected package Lexicons deploy', async () => { + const manifest = JSON.parse(await readFile(new URL('../manifest.json', import.meta.url), 'utf8')); + const { assets } = await loadAssets(fileURLToPath(new URL('../manifest.json', import.meta.url))); + const { documents } = await validatePackageLexicons(); + const validatedIds = documents.map((document) => document.id); + assert.deepEqual(validatedIds.sort(), manifest.validationLexicons.map(({ id }) => id).sort()); + + const validationSources = new Map(manifest.validationLexicons.map((source) => [source.id, source])); + const deployedPackageAssets = assets.filter(({ kind, packagePath }) => kind === 'lexicon' && packagePath); + assert.deepEqual(deployedPackageAssets.map(({ id }) => id).sort(), [ + 'app.certified.actor.organization', + 'app.certified.actor.profile', + 'app.certified.location', + 'app.certified.signature.defs', + 'org.hypercerts.defs', + ]); + for (const asset of deployedPackageAssets) { + const source = validationSources.get(asset.id); + assert.equal(asset.packagePath, source.packagePath); + const document = await readLexiconSource(source); + assert.equal(document.id, asset.id); + assert.deepEqual(asset.lexicon_json, document); + } +}); + +test('location query result refs resolve to shared actor views and getLocation-owned locationView', async () => { + const { lexicons, documents } = await validatePackageLexicons(); + const byId = new Map(documents.map((document) => [document.id, document])); + const shared = byId.get('org.hypercerts.api.defs'); + const getLocation = byId.get('app.certified.location.getLocation'); + const listLocations = byId.get('app.certified.location.listLocations'); + assert.ok(shared, 'validation sources include the shared API definitions'); + assert.ok(getLocation, 'validation sources include getLocation'); + assert.ok(listLocations, 'validation sources include listLocations'); + assert.equal(Object.hasOwn(listLocations.defs.main.parameters.properties, 'search'), false, 'listLocations must not expose a search parameter'); + + assert.equal(lexicons.getDefOrThrow('org.hypercerts.api.defs#profileView').type, 'object'); + assert.equal(lexicons.getDefOrThrow('org.hypercerts.api.defs#organizationView').type, 'object'); + assert.equal(lexicons.getDefOrThrow('org.hypercerts.api.defs#actorView').type, 'object'); + assert.deepEqual(shared.defs.profileView.required, ['uri', 'cid', 'indexedAt', 'did', 'record']); + assert.equal(shared.defs.profileView.properties.record.ref, 'lex:app.certified.actor.profile'); + assert.deepEqual(shared.defs.organizationView.required, ['uri', 'cid', 'indexedAt', 'did', 'record']); + assert.equal(shared.defs.organizationView.properties.record.ref, 'lex:app.certified.actor.organization'); + assert.deepEqual(shared.defs.actorView.required, ['did', 'profile', 'organization']); + assert.deepEqual(shared.defs.actorView.nullable, ['profile', 'organization']); + assert.equal(shared.defs.actorView.properties.profile.ref, 'lex:org.hypercerts.api.defs#profileView'); + assert.equal(shared.defs.actorView.properties.organization.ref, 'lex:org.hypercerts.api.defs#organizationView'); + assert.equal(shared.defs.locationView, undefined); + + assert.equal(getLocation.defs.locationView.type, 'object'); + assert.equal(getLocation.defs.locationView.properties.author.ref, 'lex:org.hypercerts.api.defs#actorView'); + assert.equal(getLocation.defs.locationView.properties.record.ref, 'lex:app.certified.location'); + assert.equal(getLocation.defs.output.properties.location.ref, 'lex:app.certified.location.getLocation#locationView'); + assert.equal(listLocations.defs.output.properties.locations.items.ref, 'lex:app.certified.location.getLocation#locationView'); + assert.equal(lexicons.getDefOrThrow('app.certified.location.getLocation#locationView').type, 'object'); +}); + +test('installed ATProto validator accepts package language, transitive refs, and real fixture records', async () => { + const { lexicons, isValidDid, isValidTid } = await validatePackageLexicons(); + const { jsonToLex, lexToJson } = await import('@atproto/lexicon'); + for (const record of [...locationRecords, ...profileRecords, ...organizationRecords]) { + const decoded = jsonToLex(record.record); + lexicons.assertValidRecord(record.collection, decoded); + assert.deepEqual(lexToJson(decoded), record.record); + assert.equal(isValidDid(record.did), true); + if (record.collection === 'app.certified.location') assert.equal(isValidTid(record.rkey), true); + } +}); + +test('fixture CIDs match their DAG-CBOR record contents', async () => { + const { encode } = await import('@atcute/cbor'); + const CID = await import('@atcute/cid'); + for (const record of [...locationRecords, ...profileRecords, ...organizationRecords]) { + assert.equal(CID.toString(await CID.create(0x71, encode(record.record))), record.cid); + } +}); + +test('record validator rejects malformed embedded location payloads', async () => { + const { lexicons } = await validatePackageLexicons(); + const { jsonToLex } = await import('@atproto/lexicon'); + const invalidRecord = { ...locationRecords[0].record }; + delete invalidRecord.locationType; + assert.throws(() => lexicons.assertValidRecord('app.certified.location', jsonToLex(invalidRecord)), /locationType|property/); + + const invalidBlobRecord = structuredClone(locationRecords[2].record); + invalidBlobRecord.location.blob.ref.$link = 'not-a-cid'; + assert.throws(() => lexicons.assertValidRecord('app.certified.location', jsonToLex(invalidBlobRecord)), /blob ref/); +}); diff --git a/hypercerts-api/tooling/lua.test.js b/hypercerts-api/tooling/lua.test.js new file mode 100644 index 00000000..a5b6b7a0 --- /dev/null +++ b/hypercerts-api/tooling/lua.test.js @@ -0,0 +1,51 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFile } from 'node:fs/promises'; +import { fileURLToPath } from 'node:url'; +import { spawnSync } from 'node:child_process'; +import { loadAssets } from './installer.js'; + +const root = fileURLToPath(new URL('../', import.meta.url)); +const read = (relative) => readFile(new URL(`../${relative}`, import.meta.url), 'utf8'); + +test('checked-in Lua bundles reproduce from shared and endpoint sources', async () => { + const [shared, getSource, listSource, getBeforeBuild, listBeforeBuild] = await Promise.all([ + read('lua/shared/location.lua'), read('lua/src/getLocation.lua'), read('lua/src/listLocations.lua'), + readFile(new URL('../lua/endpoints/getLocation.lua', import.meta.url)), + readFile(new URL('../lua/endpoints/listLocations.lua', import.meta.url)), + ]); + const expectedGet = Buffer.from(`${shared.trimEnd()}\n\n${getSource}`); + const expectedList = Buffer.from(`${shared.trimEnd()}\n\n${listSource}`); + assert.deepEqual(getBeforeBuild, expectedGet, 'getLocation bundle is stale; run pnpm build:lua'); + assert.deepEqual(listBeforeBuild, expectedList, 'listLocations bundle is stale; run pnpm build:lua'); + + const result = spawnSync(process.execPath, ['tooling/build-lua.js'], { cwd: root, encoding: 'utf8' }); + assert.equal(result.status, 0, result.stderr); + const [getBuilt, listBuilt] = await Promise.all([ + read('lua/endpoints/getLocation.lua'), read('lua/endpoints/listLocations.lua'), + ]); + assert.equal(getBuilt, `${shared.trimEnd()}\n\n${getSource}`); + assert.equal(listBuilt, `${shared.trimEnd()}\n\n${listSource}`); + assert.match(getBuilt, /local function get_location\(\)/); + assert.doesNotMatch(getBuilt, /local function list_locations\(\)/); + assert.match(listBuilt, /local function list_locations\(\)/); + assert.doesNotMatch(listBuilt, /local function get_location\(\)/); + assert.doesNotMatch(getBuilt, /\brequire\s*\(/); + assert.doesNotMatch(listBuilt, /\brequire\s*\(/); +}); + +test('manifest installs only built standalone handlers and records their source inputs', async () => { + const manifest = JSON.parse(await read('manifest.json')); + const { assets } = await loadAssets(fileURLToPath(new URL('../manifest.json', import.meta.url))); + assert.equal(manifest.handlerStatus.getLocation, 'implemented'); + assert.equal(manifest.handlerStatus.listLocations, 'implemented'); + assert.equal(manifest.authentication.unresolved, false); + for (const name of ['getLocation', 'listLocations']) { + const asset = assets.find(({ id }) => id === `xrpc.query:app.certified.location.${name}`); + assert.equal(asset.kind, 'script'); + assert.equal(asset.path, `../../lua/endpoints/${name}.lua`); + assert.equal(asset.sourcePath, `../../lua/src/${name}.lua`); + assert.equal(asset.sharedSourcePath, '../../lua/shared/location.lua'); + assert.match(asset.body, /function handle\(\)/); + } +}); diff --git a/hypercerts-api/tooling/seed.js b/hypercerts-api/tooling/seed.js index 88837a7b..21518022 100644 --- a/hypercerts-api/tooling/seed.js +++ b/hypercerts-api/tooling/seed.js @@ -3,14 +3,8 @@ import { accessSync, constants, statSync } from 'node:fs'; import path from 'node:path'; import { pathToFileURL } from 'node:url'; import { seedSql, locationRecords, profileRecords, organizationRecords } from '../tests/fixtures/records.js'; -import { makeDateCaseRows, badDateSeedSql } from '../tests/fixtures/bad-dates.js'; - -export const badDateLocations = await makeDateCaseRows(locationRecords[0], { - did: 'did:plc:baddatefixturesexamplexx', - decorateRecord: (record, rkey) => ({ - ...record, locationType: 'date-test', name: `Date test ${rkey}`, description: 'Synthetic timestamp fixture', - }), -}); +import { badDateSeedSql } from '../tests/fixtures/bad-dates.js'; +import { badDateLocations } from '../tests/fixtures/bad-location-dates.js'; function disposableTarget(env) { if (env.HAPPYVIEW_DISPOSABLE_TEST_TARGET !== 'YES') throw new Error('Set HAPPYVIEW_DISPOSABLE_TEST_TARGET=YES only after confirming this is a disposable test database'); diff --git a/hypercerts-api/tooling/seed.test.js b/hypercerts-api/tooling/seed.test.js index 2b988674..b433d6a3 100644 --- a/hypercerts-api/tooling/seed.test.js +++ b/hypercerts-api/tooling/seed.test.js @@ -5,7 +5,8 @@ import { chmod, copyFile, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:f import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { locationRecords, profileRecords, organizationRecords } from '../tests/fixtures/records.js'; -import { badDateLocations, buildBadDateSeedInput, buildSeedInput, psqlTargetArgs } from './seed.js'; +import { badDateLocations } from '../tests/fixtures/bad-location-dates.js'; +import { buildBadDateSeedInput, buildSeedInput, psqlTargetArgs } from './seed.js'; const packageRoot = fileURLToPath(new URL('..', import.meta.url)); @@ -17,6 +18,7 @@ async function withCopiedSeedModule(run) { await copyFile(new URL('./seed.js', import.meta.url), path.join(root, 'tooling/seed.js')); await copyFile(new URL('../tests/fixtures/records.js', import.meta.url), path.join(root, 'tests/fixtures/records.js')); await copyFile(new URL('../tests/fixtures/bad-dates.js', import.meta.url), path.join(root, 'tests/fixtures/bad-dates.js')); + await copyFile(new URL('../tests/fixtures/bad-location-dates.js', import.meta.url), path.join(root, 'tests/fixtures/bad-location-dates.js')); await writeFile(path.join(root, 'package.json'), '{"type":"module"}\n'); return await run(path.join(root, 'tooling/seed.js')); } finally { diff --git a/hypercerts-api/tooling/validate-lexicons.js b/hypercerts-api/tooling/validate-lexicons.js new file mode 100644 index 00000000..805c5d5a --- /dev/null +++ b/hypercerts-api/tooling/validate-lexicons.js @@ -0,0 +1,28 @@ +import { readFile } from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { Lexicons, isValidLexiconDoc } from '@atproto/lexicon'; +import { readLexiconSource } from './lexicon-source.js'; +import { isValidDid, isValidTid } from '@atproto/syntax'; + +export async function validatePackageLexicons(root = fileURLToPath(new URL('../', import.meta.url))) { + const manifest = JSON.parse(await readFile(path.join(root, 'manifest.json'), 'utf8')); + const documents = await Promise.all(manifest.validationLexicons.map(async (source) => { + const doc = await readLexiconSource(source, root); + if (doc.id !== source.id || !isValidLexiconDoc(doc)) throw new Error(`Invalid Lexicon v1 document or ID mismatch: ${source.packagePath ?? source.path}`); + return doc; + })); + const lexicons = new Lexicons(documents); + for (const doc of documents) { + const checkRefs = (value) => { + if (Array.isArray(value)) value.forEach(checkRefs); + else if (value && typeof value === 'object') { + if (value.type === 'ref') lexicons.getDefOrThrow(value.ref); + if (value.type === 'union') value.refs.forEach((ref) => lexicons.getDefOrThrow(ref)); + Object.values(value).forEach(checkRefs); + } + }; + checkRefs(doc); + } + return { lexicons, documents, isValidDid, isValidTid }; +}