diff --git a/README.md b/README.md index f490d64..90d60f1 100644 --- a/README.md +++ b/README.md @@ -120,22 +120,14 @@ __VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS="siva.otter-hawksbill.ts.net,.100.100.40. The content proxy preserves the tenant Host header, so the Worker still enforces its host-routing rules in development. -Production passcode login creates a seven-day, host-only -`__Host-adrive-session` cookie. That cookie is always `Secure`, `HttpOnly`, and -`SameSite=Strict`, so browsers correctly refuse it on plain-HTTP non-localhost -dev URLs (like a Tailscale hostname). Use the generated API key in the -dashboard's “Local HTTP fallback” there. Passcode login works on -`http://localhost` and on any HTTPS dashboard origin. - -Authentication bootstrap endpoints are protected by the `AUTH_GUARD` Workers -KV namespace. Login is limited to ten attempts per client every five minutes, -and five incorrect passcodes within fifteen minutes lock that client out for -thirty minutes. Device authorization creation is limited to five requests per -client every ten minutes, while device polling permits 150 requests per client -every ten minutes so the documented five-second polling interval remains valid. -Rate-limit and lockout responses use HTTP 429 with `Retry-After`. KV is -eventually consistent, so these controls are abuse mitigation rather than an -atomic global security boundary. +The dashboard signs in through WorkOS. Device authorization creation and token +polling share the `RL_AUTH` Workers rate limit: 30 requests per client address +in a 60-second window. A device polls every five seconds, leaving room for +creation and another device behind the same address. Refused token polls return +HTTP 429 with `{ "status": "slow_down" }` and `Retry-After: 60`; the CLI waits +for that delay before polling again. Creation remains rate limited and returns +the usual error response. These counters are approximate and scoped to each +Cloudflare location. `AUTH_GUARD` KV now supports caches, not auth counters. In another shell, configure and exercise the CLI: diff --git a/apps/web/.dev.vars.example b/apps/web/.dev.vars.example index cd1b516..eaeffd2 100644 --- a/apps/web/.dev.vars.example +++ b/apps/web/.dev.vars.example @@ -32,3 +32,18 @@ CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE="postgres://adrive:adri # Vite's allowlist is a SHELL setting, not a Worker variable. Start with: # __VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS="siva.otter-hawksbill.ts.net,.100.100.40.20.nip.io" bun --filter @adrive/web dev # Adjust both allowlist entries to match your dashboard/content hostnames. + +# Abuse controls (docs/abuse.md). All optional locally. +# Comma-separated WorkOS user ids allowed on /admin. The WorkOS fake signs +# you in as `user_local`, so that is the id to list here for local work. +ADMIN_USER_IDS="user_local" +# Cloudflare URL Scanner: links inside published HTML are checked against +# it. Leave empty to skip the check (recorded as such on the verdict), or +# set `fake:clean` / `fake:malicious` to exercise the pipeline offline. +# CF_ACCOUNT_ID is the account that owns the token. +URLSCAN_API_KEY="" +CF_ACCOUNT_ID="" +# Zone-level cache purge for the kill switch (Cache Purge permission on the +# content zone). Leave empty to log the purge instead. +CF_API_TOKEN="" +CF_ZONE_ID="" diff --git a/apps/web/migrations-pg/0007_scans.sql b/apps/web/migrations-pg/0007_scans.sql new file mode 100644 index 0000000..40c3dfe --- /dev/null +++ b/apps/web/migrations-pg/0007_scans.sql @@ -0,0 +1,35 @@ +-- migrate:up +-- Content scanning (services/scanner.ts). A publish by a verified org is +-- held (publish_pending) until the scanner clears it; a malicious verdict +-- on any org quarantines the file, which every content route then 404s. +-- One verdict row per (file, version, check). + +ALTER TABLE files ADD COLUMN quarantined boolean NOT NULL DEFAULT false; +ALTER TABLE files ADD COLUMN publish_pending boolean NOT NULL DEFAULT false; + +CREATE INDEX files_held_idx ON files (org_id, updated_at DESC) + WHERE quarantined OR publish_pending; + +CREATE TABLE scan_verdicts ( + file_id text NOT NULL REFERENCES files (id) ON DELETE CASCADE, + org_id text NOT NULL REFERENCES orgs (id), + version integer NOT NULL CHECK (version > 0), + verdict text NOT NULL CHECK (verdict IN ('clean', 'suspicious', 'malicious')), + source text NOT NULL, + details jsonb NOT NULL DEFAULT '{}'::jsonb, + created_at timestamptz NOT NULL DEFAULT now(), + PRIMARY KEY (file_id, version, source) +); + +CREATE INDEX scan_verdicts_review_idx ON scan_verdicts (verdict, created_at DESC) + WHERE verdict <> 'clean'; + +ALTER TABLE scan_verdicts ENABLE ROW LEVEL SECURITY; +ALTER TABLE scan_verdicts FORCE ROW LEVEL SECURITY; +CREATE POLICY org_isolation ON scan_verdicts USING (app_org_visible(org_id)); + +-- migrate:down +DROP TABLE IF EXISTS scan_verdicts; +DROP INDEX IF EXISTS files_held_idx; +ALTER TABLE files DROP COLUMN IF EXISTS publish_pending; +ALTER TABLE files DROP COLUMN IF EXISTS quarantined; diff --git a/apps/web/migrations-pg/0008_abuse.sql b/apps/web/migrations-pg/0008_abuse.sql new file mode 100644 index 0000000..4254910 --- /dev/null +++ b/apps/web/migrations-pg/0008_abuse.sql @@ -0,0 +1,51 @@ +-- migrate:up +-- Abuse controls. blocked_hashes is the first scanner check (a nightly +-- mirror of a known-bad list can append to it; so can an admin). +-- notifications tell an org's members what the scanner or an admin did to +-- their file. reports come from anyone on the content host and are worked +-- from /admin, so they are a platform table with no RLS. + +CREATE TABLE blocked_hashes ( + sha256 text PRIMARY KEY CHECK (sha256 ~ '^[0-9a-f]{64}$'), + reason text NOT NULL DEFAULT '', + added_by text, + added_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE notifications ( + id text PRIMARY KEY, + org_id text NOT NULL REFERENCES orgs (id) ON DELETE CASCADE, + kind text NOT NULL, + message text NOT NULL, + file_id text, + created_at timestamptz NOT NULL DEFAULT now(), + read_at timestamptz +); + +CREATE INDEX notifications_unread_idx ON notifications (org_id, created_at DESC) + WHERE read_at IS NULL; + +ALTER TABLE notifications ENABLE ROW LEVEL SECURITY; +ALTER TABLE notifications FORCE ROW LEVEL SECURITY; +CREATE POLICY org_isolation ON notifications USING (app_org_visible(org_id)); + +CREATE TABLE reports ( + id text PRIMARY KEY, + org_id text NOT NULL, + file_id text NOT NULL, + version integer, + reason text NOT NULL, + reporter_ip_hash text NOT NULL, + details text, + created_at timestamptz NOT NULL DEFAULT now(), + resolved_at timestamptz, + resolution text +); + +CREATE INDEX reports_open_idx ON reports (created_at DESC) WHERE resolved_at IS NULL; +CREATE INDEX reports_file_idx ON reports (file_id, created_at DESC); + +-- migrate:down +DROP TABLE IF EXISTS reports; +DROP TABLE IF EXISTS notifications; +DROP TABLE IF EXISTS blocked_hashes; diff --git a/apps/web/migrations-pg/0009_scan_recovery.sql b/apps/web/migrations-pg/0009_scan_recovery.sql new file mode 100644 index 0000000..258fefd --- /dev/null +++ b/apps/web/migrations-pg/0009_scan_recovery.sql @@ -0,0 +1,13 @@ +-- migrate:up +-- Persist scan work with the version write so a lost post-commit queue +-- send can be recovered. The timestamp also fences a completed scan +-- from clearing a newer request for the same immutable version. +ALTER TABLE file_versions ADD COLUMN scan_next_run_at timestamptz; + +CREATE INDEX file_versions_scan_due_idx + ON file_versions (org_id, scan_next_run_at, file_id, version) + WHERE scan_next_run_at IS NOT NULL; + +-- migrate:down +DROP INDEX IF EXISTS file_versions_scan_due_idx; +ALTER TABLE file_versions DROP COLUMN IF EXISTS scan_next_run_at; diff --git a/apps/web/package.json b/apps/web/package.json index 7589b52..b31ab5b 100644 --- a/apps/web/package.json +++ b/apps/web/package.json @@ -22,6 +22,7 @@ "effect": "4.0.0-beta.102", "markdown-it": "^14.3.0", "mime": "^4.1.0", + "parse5": "^8.0.1", "pg": "^8.22.0", "runed": "^0.37.1", "zod": "^4.4.3" diff --git a/apps/web/src/app.d.ts b/apps/web/src/app.d.ts index 127db9b..bb8df55 100644 --- a/apps/web/src/app.d.ts +++ b/apps/web/src/app.d.ts @@ -7,7 +7,7 @@ declare global { // Set by the root layout load for every page. interface PageData { session?: { - readonly user: { readonly email: string }; + readonly user: { readonly email: string; readonly admin: boolean }; readonly role: AuthContext['role']; readonly org: { readonly name: string; readonly slug: string }; } | null; diff --git a/apps/web/src/env.d.ts b/apps/web/src/env.d.ts index 83d54a0..d78c29d 100644 --- a/apps/web/src/env.d.ts +++ b/apps/web/src/env.d.ts @@ -9,6 +9,13 @@ declare global { WORKOS_CLIENT_ID?: string; WORKOS_COOKIE_PASSWORD?: string; WORKOS_WEBHOOK_SECRET?: string; + // Abuse controls (docs/abuse.md). All optional: the Null services + // stand in when unset. + ADMIN_USER_IDS?: string; + URLSCAN_API_KEY?: string; + CF_ACCOUNT_ID?: string; + CF_API_TOKEN?: string; + CF_ZONE_ID?: string; } namespace Cloudflare { @@ -19,6 +26,11 @@ declare global { WORKOS_CLIENT_ID?: string; WORKOS_COOKIE_PASSWORD?: string; WORKOS_WEBHOOK_SECRET?: string; + ADMIN_USER_IDS?: string; + URLSCAN_API_KEY?: string; + CF_ACCOUNT_ID?: string; + CF_API_TOKEN?: string; + CF_ZONE_ID?: string; } } } diff --git a/apps/web/src/lib/components/Dashboard.svelte b/apps/web/src/lib/components/Dashboard.svelte index 9229048..0ee08dd 100644 --- a/apps/web/src/lib/components/Dashboard.svelte +++ b/apps/web/src/lib/components/Dashboard.svelte @@ -152,18 +152,26 @@ }; const openFile = async (file: DashboardFile) => { + const opensTab = file.public || file.kind === 'site'; + const tab = opensTab ? window.open('about:blank', '_blank') : null; + if (tab) tab.opener = null; + if (opensTab && !tab) { + toasts.info('Allow pop-ups to open this file'); + return; + } try { const url = await resolveFileLink( file, session.token, files.list.current.contentOrigin ); - if (file.public) { - window.open(url, '_blank', 'noopener'); + if (tab) { + tab.location.replace(url); } else { download(url, file.displayName); } } catch (cause) { + tab?.close(); toasts.error(cause, 'Could not open the file'); } }; diff --git a/apps/web/src/lib/components/FileDetailView.svelte b/apps/web/src/lib/components/FileDetailView.svelte index ccedb2f..6917fd9 100644 --- a/apps/web/src/lib/components/FileDetailView.svelte +++ b/apps/web/src/lib/components/FileDetailView.svelte @@ -130,7 +130,13 @@ return false; } } - toasts.success(result.forcedPublic ? 'HTML files stay public' : success); + toasts.success( + mutation.action === 'visibility' && result.file.publishPending + ? 'Pending review before public sharing' + : result.forcedPublic + ? 'HTML files stay public' + : success + ); return true; } catch (cause) { if ( @@ -155,7 +161,10 @@ const linkFor = async (version?: number) => { const current = detail.current; - if (!current) return { url: '', expiresAt: null }; + if (!current) return { url: '', expiresAt: null, public: false }; + if (current.file.quarantined) { + throw new Error('Sharing is unavailable while this file is quarantined'); + } const fileId = current.file.id; const expirationTime = current.file.expiresAt ? new Date(current.file.expiresAt).getTime() @@ -163,9 +172,13 @@ const unavailable = current.file.deletedAt !== null || (Number.isFinite(expirationTime) && expirationTime <= Date.now()); + // Historical bytes have their own scan clearance. Resolve those + // through the API so an owner grant can cover an unreviewed version. + const currentVersion = + version === undefined || version === current.file.version; const link = - current.file.public && !unavailable - ? { url: fileUrl(version), expiresAt: null } + current.file.public && !unavailable && currentVersion + ? { url: fileUrl(version), expiresAt: null, public: true } : getContentLink( session.token, current.file.id, @@ -189,7 +202,7 @@ const fileId = current.file.id; const link = await linkFor(version); if (detail.current?.file.id !== fileId) return; - if (current.file.public) { + if (link.public || current.file.kind === 'site') { window.open(link.url, '_blank', 'noopener'); } else { const anchor = document.createElement('a'); @@ -274,7 +287,7 @@ const putVersion = async (file: File) => { const current = detail.current; - if (!current || busy) return; + if (!current || busy || current.file.quarantined) return; const fileId = current.file.id; const currentOperation = ++operation; if (file.size > current.maxUploadBytes) { @@ -347,27 +360,50 @@ Files - - void update({ action: 'rename', displayName }, 'File renamed')} - /> + {#if detail.current.file.quarantined} +

+ {detail.current.file.displayName} +

+ {:else} + + void update({ action: 'rename', displayName }, 'File renamed')} + /> + {/if} {#if detail.current.file.deletedAt} In trash {/if} - resolveLink()} /> + resolveLink()} + disabled={detail.current.file.quarantined} + />
- void openLink()} - /> + {#if detail.current.file.quarantined} +
+

+ Preview unavailable while this file is quarantined. +

+
+ {:else} + void openLink()} + /> + {/if} import type { DashboardFile } from '@adrive/shared'; import { formatBytes, formatDate } from '$lib/dashboard/format'; + import Icon from '$lib/components/ui/Icon.svelte'; import FileMenu from './FileMenu.svelte'; import FileThumb from './FileThumb.svelte'; @@ -39,14 +40,22 @@
  • - + {#if file.quarantined} +
    + +
    + {:else} + + {/if}
    {#if onselect} @@ -86,12 +95,22 @@
    - - {file.public ? 'Public' : 'Private'} + + {file.quarantined + ? 'Quarantined' + : file.publishPending + ? 'Pending review' + : file.public + ? 'Public' + : 'Private'} {#each file.tags.slice(0, 2) as tag (tag.id)} Tags Size Modified - Access + Access Actions @@ -72,8 +72,14 @@ {formatDate(file.updatedAt)} - - {file.public ? 'Public' : 'Private'} + + {file.quarantined + ? 'Quarantined' + : file.publishPending + ? 'Pending review' + : file.public + ? 'Public' + : 'Private'} @@ -39,8 +39,14 @@ > {/if} {:else} -
    {/if}
    diff --git a/apps/web/src/lib/dashboard/parse.ts b/apps/web/src/lib/dashboard/parse.ts index 73e1c06..55403ae 100644 --- a/apps/web/src/lib/dashboard/parse.ts +++ b/apps/web/src/lib/dashboard/parse.ts @@ -130,6 +130,8 @@ const parseDashboardFile = (value: unknown, path = 'file'): DashboardFile => { const record = requireRecord(value, path); return { ...parseFileSummary(value, path), + quarantined: flag(record.quarantined, `${path}.quarantined`), + publishPending: flag(record.publishPending, `${path}.publishPending`), deletedAt: maybeString(record.deletedAt, `${path}.deletedAt`), updatedAt: text(record.updatedAt, `${path}.updatedAt`), htmlForcedPublic: flag(record.htmlForcedPublic, `${path}.htmlForcedPublic`), diff --git a/apps/web/src/lib/server/auth-rate-limit-response.ts b/apps/web/src/lib/server/auth-rate-limit-response.ts index 57766ec..54523de 100644 --- a/apps/web/src/lib/server/auth-rate-limit-response.ts +++ b/apps/web/src/lib/server/auth-rate-limit-response.ts @@ -1,8 +1,9 @@ -import type { BlockedAuthAttempt } from './services/auth-guard'; +// Every rate limit binding counts over a 60 second window +// (wrangler.jsonc), so a refused request can try again after it. +export const RATE_LIMIT_PERIOD_SECONDS = 60; -export const authRateLimitResponse = ( - decision: BlockedAuthAttempt, - message = 'Too many authentication requests. Try again later.' +export const rateLimitResponse = ( + message = 'Too many requests. Try again later.' ) => Response.json( { message }, @@ -10,7 +11,7 @@ export const authRateLimitResponse = ( status: 429, headers: { 'Cache-Control': 'private, no-store', - 'Retry-After': String(decision.retryAfterSeconds) + 'Retry-After': String(RATE_LIMIT_PERIOD_SECONDS) } } ); diff --git a/apps/web/src/lib/server/config.ts b/apps/web/src/lib/server/config.ts index 9b17749..47968ae 100644 --- a/apps/web/src/lib/server/config.ts +++ b/apps/web/src/lib/server/config.ts @@ -11,6 +11,21 @@ export interface WorkOSConfig { readonly webhookSecret: string; } +// Cloudflare URL Scanner (services/url-reputation.ts). Null when +// URLSCAN_API_KEY is unset; a `fake:` key selects the fake only +// in development. Production rejects fake keys. +export interface UrlScannerConfig { + readonly apiKey: string; + readonly accountId: string; +} + +// Zone-level cache purge (services/cache-purge.ts). Null when either +// CF_API_TOKEN or CF_ZONE_ID is unset. +export interface CloudflareZoneConfig { + readonly apiToken: string; + readonly zoneId: string; +} + export interface AppConfigShape { readonly dashboardOrigin: string; // Tenant content is served from `.` over the @@ -22,6 +37,10 @@ export interface AppConfigShape { // Signs the Worker facade's cron and queue self-requests. readonly maintenanceSecret: string; readonly workos: WorkOSConfig; + readonly urlScanner: UrlScannerConfig | null; + readonly cloudflareZone: CloudflareZoneConfig | null; + // WorkOS user ids allowed on /admin (ADMIN_USER_IDS, comma-separated). + readonly adminUserIds: ReadonlySet; readonly semanticSearch: 'off' | 'auto' | 'required'; readonly embeddingModel: '@cf/baai/bge-small-en-v1.5'; readonly embeddingPooling: 'cls'; @@ -78,6 +97,33 @@ const workosFromEnv = (env: Env): WorkOSConfig => { return { apiKey: rawApiKey, clientId, cookiePassword, webhookSecret }; }; +const urlScannerFromEnv = (env: Env): UrlScannerConfig | null => { + const apiKey = optionalString(env.URLSCAN_API_KEY).trim(); + if (apiKey === '') return null; + if (apiKey.startsWith('fake:') && !dev) { + throw new Error('Fake URL scanning is only available in development'); + } + const accountId = optionalString(env.CF_ACCOUNT_ID); + if (!accountId && !apiKey.startsWith('fake:')) { + throw new Error('CF_ACCOUNT_ID is required alongside URLSCAN_API_KEY'); + } + return { apiKey, accountId }; +}; + +const cloudflareZoneFromEnv = (env: Env): CloudflareZoneConfig | null => { + const apiToken = optionalString(env.CF_API_TOKEN); + const zoneId = optionalString(env.CF_ZONE_ID); + return apiToken && zoneId ? { apiToken, zoneId } : null; +}; + +const adminUserIdsFromEnv = (env: Env) => + new Set( + optionalString(env.ADMIN_USER_IDS) + .split(',') + .map((id) => id.trim()) + .filter((id) => id.length > 0) + ); + export const configFromEnv = (env: Env) => { const origins = normalizeOrigins({ dashboardOrigin: env.DASHBOARD_ORIGIN, @@ -111,6 +157,9 @@ export const configFromEnv = (env: Env) => { maxUploadBytes, maintenanceSecret: env.MAINTENANCE_SECRET, workos: workosFromEnv(env), + urlScanner: urlScannerFromEnv(env), + cloudflareZone: cloudflareZoneFromEnv(env), + adminUserIds: adminUserIdsFromEnv(env), semanticSearch, embeddingModel: '@cf/baai/bge-small-en-v1.5', embeddingPooling: 'cls', diff --git a/apps/web/src/lib/server/content-host.ts b/apps/web/src/lib/server/content-host.ts index 58615b5..c786734 100644 --- a/apps/web/src/lib/server/content-host.ts +++ b/apps/web/src/lib/server/content-host.ts @@ -25,6 +25,8 @@ export type ContentHostResolution = // of requests for a random host does not fan out to Postgres. Suspending // an org or changing its slug purges the key (forgetContentSlug) to speed // propagation; the TTL bounds recovery when a purge cannot be delivered. +// Cached identities are checked against current Postgres trust before use: KV +// invalidation alone cannot revoke access consistently across locations. export const contentSlugCacheKey = (slug: string) => `org-slug:${slug}`; const CACHE_TTL_SECONDS = 300; const NEGATIVE_CACHE_TTL_SECONDS = 60; @@ -69,7 +71,23 @@ export const resolveContentSlug = Effect.fn('resolveContentSlug')(function* ( new StorageError({ operation: 'read slug cache', cause }) }) ); - if (cached) return resolution(slug, cached); + if (cached && !('orgId' in cached)) return resolution(slug, cached); + if (cached && 'orgId' in cached) { + const current = yield* sql<{ trust: string }>` + SELECT trust FROM orgs WHERE id = ${cached.orgId} AND slug = ${slug} + `.pipe( + Effect.mapError( + (cause) => + new StorageError({ + operation: 'check content org trust', + cause + }) + ) + ); + if (current[0]) + return resolution(slug, { orgId: cached.orgId, trust: current[0].trust }); + // A stale mapping after rename/deletion falls through to live resolution. + } // The live slug wins; otherwise a slug the org released within the // redirect window points at its current one (a suspended org's old // slug still redirects, to a host that then answers 404). diff --git a/apps/web/src/lib/server/content-version-access.ts b/apps/web/src/lib/server/content-version-access.ts new file mode 100644 index 0000000..46cded1 --- /dev/null +++ b/apps/web/src/lib/server/content-version-access.ts @@ -0,0 +1,31 @@ +import type { PgClient } from '@effect/sql-pg'; + +// Used by both file and thumbnail lookups, before consulting any cache. +// Public status belongs to the current file, but it does not clear older +// bytes that may never have been scanned. Owner grants can still reach +// those versions; a malicious verdict excludes a version even with a grant. +export const contentVersionAccess = (sql: PgClient.PgClient) => ({ + review: sql` + LEFT JOIN LATERAL ( + SELECT + max(verdict) FILTER (WHERE source = 'admin') AS admin_verdict, + count(*) FILTER ( + WHERE source IN ('hash', 'sniff', 'urlscan') AND verdict = 'clean' + ) = 3 AND NOT COALESCE(bool_or(verdict <> 'clean') + FILTER (WHERE source <> 'admin'), false) AS scanned_clean, + COALESCE(bool_or(verdict = 'malicious') + FILTER (WHERE source <> 'admin'), false) AS malicious + FROM scan_verdicts + WHERE file_id = f.id AND org_id = f.org_id AND version = v.version + ) scan_access ON true + `, + allowed: sql`COALESCE(scan_access.admin_verdict <> 'malicious', NOT scan_access.malicious)`, + isPublic: sql`f.public AND ( + v.version = f.current_version + OR COALESCE(scan_access.admin_verdict = 'clean', false) + OR ( + scan_access.admin_verdict IS NULL AND scan_access.scanned_clean + AND v.scan_next_run_at IS NULL + ) + )` +}); diff --git a/apps/web/src/lib/server/edge.ts b/apps/web/src/lib/server/edge.ts index 521428c..3d2d4d4 100644 --- a/apps/web/src/lib/server/edge.ts +++ b/apps/web/src/lib/server/edge.ts @@ -19,7 +19,6 @@ import type { ProgramIdentity, ProgramTenant } from './identity'; import { requestLayer } from './layer'; import { PgSql } from './pg'; import type { AppConfig } from './config'; -import type { AuthGuard } from './services/auth-guard'; import type { Auth } from './services/auth'; import type { AuthGuardStore } from './services/bindings'; import type { Blobs } from './services/blobs'; @@ -34,12 +33,21 @@ import type { Lifecycle } from './services/lifecycle'; import type { GrantSecrets } from './services/grant-secrets'; import type { JobQueue } from './services/jobs'; import type { Org } from './services/org'; +import type { RateLimits } from './services/rate-limits'; +import type { Admin } from './services/admin'; +import type { CloudflareCachePurge } from './services/cache-purge'; +import type { Scanner } from './services/scanner'; +import type { UrlReputation } from './services/url-reputation'; import type { WorkOSClient } from './services/workos'; export type AppServices = | PgSql | AppConfig - | AuthGuard + | RateLimits + | UrlReputation + | CloudflareCachePurge + | Scanner + | Admin | AuthGuardStore | Auth | Blobs diff --git a/apps/web/src/lib/server/file-content-link.ts b/apps/web/src/lib/server/file-content-link.ts index cc66fd4..e30abaf 100644 --- a/apps/web/src/lib/server/file-content-link.ts +++ b/apps/web/src/lib/server/file-content-link.ts @@ -73,7 +73,7 @@ export const resolveFileContentLink = ( message: 'Only the current site version can be previewed' }); } - if (!includeUnavailable && !requireGrant) { + if (resolved.file.public && !includeUnavailable && !requireGrant) { return { url: new URL( `/s/${encodeURIComponent(resolved.file.id)}/`, diff --git a/apps/web/src/lib/server/file-rows.ts b/apps/web/src/lib/server/file-rows.ts index 8b53bce..4cda309 100644 --- a/apps/web/src/lib/server/file-rows.ts +++ b/apps/web/src/lib/server/file-rows.ts @@ -9,6 +9,8 @@ export const DashboardFileRow = Schema.Struct({ current_version: Schema.Int, size_bytes: Schema.Int, is_public: Schema.Boolean, + quarantined: Schema.Boolean, + publish_pending: Schema.Boolean, has_html: Schema.Boolean, created_at: Schema.String, updated_at: Schema.String, @@ -40,6 +42,8 @@ export const dashboardFileColumns = ` f.current_version, f.size_bytes, f.public AS is_public, + f.quarantined, + f.publish_pending, EXISTS ( SELECT 1 FROM file_versions html_version WHERE html_version.file_id = f.id @@ -87,6 +91,8 @@ export const toDashboardFile = ( version: row.current_version, sizeBytes: row.size_bytes, public: row.is_public, + quarantined: row.quarantined, + publishPending: row.publish_pending, htmlForcedPublic: row.has_html, createdAt: row.created_at, updatedAt: row.updated_at, diff --git a/apps/web/src/lib/server/host-gate.ts b/apps/web/src/lib/server/host-gate.ts index ae0e820..bde67af 100644 --- a/apps/web/src/lib/server/host-gate.ts +++ b/apps/web/src/lib/server/host-gate.ts @@ -73,13 +73,16 @@ export const normalizeOrigins = (config: OriginConfig) => { }; }; +// Content hosts serve files, sites, thumbnails, and the abuse report form +// for what they serve. export const classifyRoute = (pathname: string) => pathname === '/f' || pathname.startsWith('/f/') || pathname === '/t' || pathname.startsWith('/t/') || pathname === '/s' || - pathname.startsWith('/s/') + pathname.startsWith('/s/') || + pathname === '/report' ? 'content' : 'dashboard'; diff --git a/apps/web/src/lib/server/html-links.ts b/apps/web/src/lib/server/html-links.ts new file mode 100644 index 0000000..01d8247 --- /dev/null +++ b/apps/web/src/lib/server/html-links.ts @@ -0,0 +1,90 @@ +import { parse, type DefaultTreeAdapterMap } from 'parse5'; + +// The outbound targets inside a published HTML document: ``, +// ` + + + Admin · adrive + + +
    +

    Admin

    + +
    +

    + Reports ({overview.reports.length}) +

    +
    + + + + + + + + + + + + + {#each overview.reports as report (report.id)} + + + + + + + + + {:else} + + {/each} + +
    FiledReasonFileOrgReporter
    {formatDate(report.createdAt)} + {report.reason} + {#if report.details} +
    + {report.details} +
    + {/if} +
    + {#if report.file} + {report.file.name} +
    + v{report.file.version} · {report.file.quarantined + ? 'quarantined' + : report.file.publishPending + ? 'held' + : report.file.public + ? 'public' + : 'private'} +
    + {:else} + gone + {/if} +
    + {report.fileId} +
    +
    {report.orgSlug ?? report.orgId}{report.reporter} +
    + + +
    { + event.preventDefault(); + const value = new FormData(event.currentTarget).get( + 'resolution' + ); + const resolution = resolutions.find( + (item) => item === value + ); + if (resolution) void resolveReport(report.id, resolution); + }} + > + + +
    +
    +
    None
    +
    +
    + +
    +

    + Held and quarantined files ({overview.held.length}) +

    +
    + + + + + + + + + + + + + {#each overview.held as file (file.id)} + + + + + + + + + {:else} + + {/each} + +
    UpdatedFileOrgStateVerdicts
    {formatDate(file.updatedAt)} + {file.name} +
    + {file.contentType} · v{file.version} +
    +
    {file.id}
    +
    {file.orgSlug} + {file.quarantined + ? 'quarantined' + : file.publishPending + ? 'held' + : 'live, flagged'} + + {#each file.verdicts as verdict (verdict.source)} +
    + + {verdict.source} + {verdict.verdict} + +
    {JSON.stringify(
    +												verdict.details,
    +												null,
    +												2
    +											)}
    +
    + {/each} +
    +
    + + +
    +
    None
    +
    +
    + +
    +

    + Failed jobs ({overview.failedJobs.length}) +

    +
    + + + + + + + + + + + + {#each overview.failedJobs as job (job.id)} + + + + + + + + {:else} + + {/each} + +
    FailedKindOrgAttemptsError
    {formatDate(job.failedAt)} + {job.kind} +
    + {JSON.stringify(job.payload)} +
    +
    {job.orgId ?? ''}{job.attempts}{job.error}
    None
    +
    +
    + +
    +

    + Recent orgs ({overview.orgs.length}) +

    +
    + + + + + + + + + + + + + + + {#each overview.orgs as org (org.id)} + + + + + + + + + + + {:else} + + {/each} + +
    CreatedSlugIdTrustPlanStoredFiles
    {formatDate(org.createdAt)}{org.slug}{org.id}{org.trust}{org.plan}{formatBytes(org.storedBytes)}{org.fileCount} +
    + {#if org.trust === 'suspended'} + + {:else} + {#if org.trust !== 'established'} + + {/if} + + {/if} +
    +
    None
    +
    +
    + +
    +

    Block a hash

    +
    { + event.preventDefault(); + void blockHash(); + }} + > + + + +
    +
    +
    diff --git a/apps/web/src/routes/api/admin/files/[id]/+server.ts b/apps/web/src/routes/api/admin/files/[id]/+server.ts new file mode 100644 index 0000000..c548a69 --- /dev/null +++ b/apps/web/src/routes/api/admin/files/[id]/+server.ts @@ -0,0 +1,27 @@ +import type { RequestHandler } from './$types'; +import { Effect, Schema } from 'effect'; +import { runEdge } from '$lib/server/edge'; +import { decodeJson } from '$lib/server/request-json'; +import { requireAdmin } from '$lib/server/request-auth'; +import { Admin } from '$lib/server/services/admin'; + +const Body = Schema.Struct({ + verdict: Schema.Literals(['clean', 'malicious']), + version: Schema.Int +}); + +// An operator's verdict on a held or quarantined file. +export const PATCH: RequestHandler = (event) => + runEdge( + Effect.gen(function* () { + const admin = yield* Admin; + const auth = yield* requireAdmin(event); + const { verdict, version } = yield* decodeJson( + event.request, + Body, + 'A verdict (clean, malicious) is required' + ); + yield* admin.markFile(event.params.id, version, verdict, auth.userId); + return Response.json({ ok: true as const }); + }) + ); diff --git a/apps/web/src/routes/api/admin/hashes/+server.ts b/apps/web/src/routes/api/admin/hashes/+server.ts new file mode 100644 index 0000000..b1751ec --- /dev/null +++ b/apps/web/src/routes/api/admin/hashes/+server.ts @@ -0,0 +1,28 @@ +import type { RequestHandler } from './$types'; +import { Effect, Schema } from 'effect'; +import { runEdge } from '$lib/server/edge'; +import { decodeJson } from '$lib/server/request-json'; +import { requireAdmin } from '$lib/server/request-auth'; +import { Admin } from '$lib/server/services/admin'; + +const Body = Schema.Struct({ + sha256: Schema.String, + reason: Schema.optionalKey(Schema.String) +}); + +// Appends to blocked_hashes; the scanner quarantines any upload that +// matches from then on. +export const POST: RequestHandler = (event) => + runEdge( + Effect.gen(function* () { + const admin = yield* Admin; + const auth = yield* requireAdmin(event); + const body = yield* decodeJson( + event.request, + Body, + 'A sha256 is required' + ); + yield* admin.blockHash(body.sha256, body.reason ?? '', auth.userId); + return Response.json({ ok: true as const }, { status: 201 }); + }) + ); diff --git a/apps/web/src/routes/api/admin/orgs/[id]/+server.ts b/apps/web/src/routes/api/admin/orgs/[id]/+server.ts new file mode 100644 index 0000000..ae4056f --- /dev/null +++ b/apps/web/src/routes/api/admin/orgs/[id]/+server.ts @@ -0,0 +1,36 @@ +import type { RequestHandler } from './$types'; +import { Effect, Schema } from 'effect'; +import { runEdge } from '$lib/server/edge'; +import { decodeJson } from '$lib/server/request-json'; +import { requireAdmin } from '$lib/server/request-auth'; +import { Admin } from '$lib/server/services/admin'; + +const Body = Schema.Union([ + Schema.Struct({ action: Schema.Literal('suspend') }), + Schema.Struct({ action: Schema.Literal('restore') }), + Schema.Struct({ + action: Schema.Literal('trust'), + trust: Schema.Literals(['new', 'verified', 'established']) + }) +]); + +export const PATCH: RequestHandler = (event) => + runEdge( + Effect.gen(function* () { + const admin = yield* Admin; + yield* requireAdmin(event); + const body = yield* decodeJson( + event.request, + Body, + 'An org action (suspend, restore, trust) is required' + ); + const orgId = event.params.id; + const org = + body.action === 'suspend' + ? yield* admin.suspendOrg(orgId) + : body.action === 'restore' + ? yield* admin.restoreOrg(orgId) + : yield* admin.setTrust(orgId, body.trust); + return Response.json({ org }); + }) + ); diff --git a/apps/web/src/routes/api/admin/overview/+server.ts b/apps/web/src/routes/api/admin/overview/+server.ts new file mode 100644 index 0000000..92da4c8 --- /dev/null +++ b/apps/web/src/routes/api/admin/overview/+server.ts @@ -0,0 +1,14 @@ +import type { RequestHandler } from './$types'; +import { Effect } from 'effect'; +import { runEdge } from '$lib/server/edge'; +import { requireAdmin } from '$lib/server/request-auth'; +import { Admin } from '$lib/server/services/admin'; + +export const GET: RequestHandler = (event) => + runEdge( + Effect.gen(function* () { + const admin = yield* Admin; + yield* requireAdmin(event); + return Response.json(yield* admin.overview); + }) + ); diff --git a/apps/web/src/routes/api/admin/reports/[id]/+server.ts b/apps/web/src/routes/api/admin/reports/[id]/+server.ts new file mode 100644 index 0000000..e2a1094 --- /dev/null +++ b/apps/web/src/routes/api/admin/reports/[id]/+server.ts @@ -0,0 +1,24 @@ +import type { RequestHandler } from './$types'; +import { Effect, Schema } from 'effect'; +import { runEdge } from '$lib/server/edge'; +import { RESOLUTIONS } from '$lib/server/report-policy'; +import { decodeJson } from '$lib/server/request-json'; +import { requireAdmin } from '$lib/server/request-auth'; +import { Admin } from '$lib/server/services/admin'; + +const Body = Schema.Struct({ resolution: Schema.Literals(RESOLUTIONS) }); + +export const PATCH: RequestHandler = (event) => + runEdge( + Effect.gen(function* () { + const admin = yield* Admin; + yield* requireAdmin(event); + const { resolution } = yield* decodeJson( + event.request, + Body, + `A resolution (${RESOLUTIONS.join(', ')}) is required` + ); + yield* admin.resolveReport(event.params.id, resolution); + return Response.json({ ok: true as const }); + }) + ); diff --git a/apps/web/src/routes/api/auth/device/+server.ts b/apps/web/src/routes/api/auth/device/+server.ts index 3fa6556..f85d64b 100644 --- a/apps/web/src/routes/api/auth/device/+server.ts +++ b/apps/web/src/routes/api/auth/device/+server.ts @@ -1,24 +1,25 @@ import { DeviceAuthorizationCreateSchema } from '@adrive/shared'; import type { RequestHandler } from './$types'; import { Effect } from 'effect'; -import { authRateLimitResponse } from '$lib/server/auth-rate-limit-response'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { AppConfig } from '$lib/server/config'; import { runEdge } from '$lib/server/edge'; import { decodeJson } from '$lib/server/request-json'; import { Auth } from '$lib/server/services/auth'; -import { AuthGuard } from '$lib/server/services/auth-guard'; +import { RateLimits } from '$lib/server/services/rate-limits'; export const POST: RequestHandler = ({ request, getClientAddress }) => runEdge( Effect.gen(function* () { const auth = yield* Auth; - const authGuard = yield* AuthGuard; + const rateLimits = yield* RateLimits; const config = yield* AppConfig; - const rateLimit = yield* authGuard.consume( - 'deviceCreate', - getClientAddress() - ); - if (!rateLimit.allowed) return authRateLimitResponse(rateLimit); + const rateLimit = yield* rateLimits.auth(getClientAddress()); + if (!rateLimit.allowed) { + return rateLimitResponse( + 'Too many authentication requests. Try again later.' + ); + } const input = yield* decodeJson( request, DeviceAuthorizationCreateSchema, diff --git a/apps/web/src/routes/api/auth/device/token/+server.ts b/apps/web/src/routes/api/auth/device/token/+server.ts index cbc7d77..9e85869 100644 --- a/apps/web/src/routes/api/auth/device/token/+server.ts +++ b/apps/web/src/routes/api/auth/device/token/+server.ts @@ -1,22 +1,30 @@ import { DeviceTokenRequestSchema } from '@adrive/shared'; import type { RequestHandler } from './$types'; import { Effect } from 'effect'; -import { authRateLimitResponse } from '$lib/server/auth-rate-limit-response'; +import { RATE_LIMIT_PERIOD_SECONDS } from '$lib/server/auth-rate-limit-response'; import { runEdge } from '$lib/server/edge'; import { decodeJson } from '$lib/server/request-json'; import { Auth } from '$lib/server/services/auth'; -import { AuthGuard } from '$lib/server/services/auth-guard'; +import { RateLimits } from '$lib/server/services/rate-limits'; export const POST: RequestHandler = ({ request, getClientAddress }) => runEdge( Effect.gen(function* () { const auth = yield* Auth; - const authGuard = yield* AuthGuard; - const rateLimit = yield* authGuard.consume( - 'devicePoll', - getClientAddress() - ); - if (!rateLimit.allowed) return authRateLimitResponse(rateLimit); + const rateLimits = yield* RateLimits; + const rateLimit = yield* rateLimits.auth(getClientAddress()); + if (!rateLimit.allowed) { + return Response.json( + { status: 'slow_down' }, + { + status: 429, + headers: { + 'Cache-Control': 'private, no-store', + 'Retry-After': String(RATE_LIMIT_PERIOD_SECONDS) + } + } + ); + } const input = yield* decodeJson( request, DeviceTokenRequestSchema, diff --git a/apps/web/src/routes/api/files/+server.ts b/apps/web/src/routes/api/files/+server.ts index edf5e5e..6aca304 100644 --- a/apps/web/src/routes/api/files/+server.ts +++ b/apps/web/src/routes/api/files/+server.ts @@ -4,10 +4,10 @@ import { runEdge } from '$lib/server/edge'; import { requireAuth, requireWrite } from '$lib/server/request-auth'; import { AppConfig } from '$lib/server/config'; import { validateExpiration } from '$lib/server/auth-policy'; -import { authRateLimitResponse } from '$lib/server/auth-rate-limit-response'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { InvalidRequest } from '$lib/server/errors'; import { parsePageSize } from '$lib/server/list-cursor'; -import { AuthGuard } from '$lib/server/services/auth-guard'; +import { RateLimits } from '$lib/server/services/rate-limits'; import { currentContentOrigin } from '$lib/server/services/current-org'; import { Files } from '$lib/server/services/files'; import { Indexing } from '$lib/server/services/indexing'; @@ -120,18 +120,12 @@ export const PUT: RequestHandler = (event) => { const { request, url } = event; return runEdge( Effect.gen(function* () { - const authGuard = yield* AuthGuard; + const rateLimits = yield* RateLimits; const files = yield* Files; const credential = yield* requireWrite(event); - const rateLimit = yield* authGuard.consume( - 'upload', - credential.credentialId - ); + const rateLimit = yield* rateLimits.upload(credential.orgId); if (!rateLimit.allowed) { - return authRateLimitResponse( - rateLimit, - 'Too many uploads. Try again later.' - ); + return rateLimitResponse('Too many uploads. Try again later.'); } const displayName = yield* Effect.try({ try: () => decodeName(request.headers.get('x-adrive-file-name')), diff --git a/apps/web/src/routes/api/files/[id]/versions/+server.ts b/apps/web/src/routes/api/files/[id]/versions/+server.ts index 27a3857..fb84b68 100644 --- a/apps/web/src/routes/api/files/[id]/versions/+server.ts +++ b/apps/web/src/routes/api/files/[id]/versions/+server.ts @@ -1,27 +1,21 @@ import type { RequestHandler } from './$types'; import { Effect } from 'effect'; -import { authRateLimitResponse } from '$lib/server/auth-rate-limit-response'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { runEdge } from '$lib/server/edge'; import { requireWrite } from '$lib/server/request-auth'; -import { AuthGuard } from '$lib/server/services/auth-guard'; +import { RateLimits } from '$lib/server/services/rate-limits'; import { Files } from '$lib/server/services/files'; export const PUT: RequestHandler = (event) => { const { params, request } = event; return runEdge( Effect.gen(function* () { - const authGuard = yield* AuthGuard; + const rateLimits = yield* RateLimits; const files = yield* Files; const credential = yield* requireWrite(event); - const rateLimit = yield* authGuard.consume( - 'upload', - credential.credentialId - ); + const rateLimit = yield* rateLimits.upload(credential.orgId); if (!rateLimit.allowed) { - return authRateLimitResponse( - rateLimit, - 'Too many uploads. Try again later.' - ); + return rateLimitResponse('Too many uploads. Try again later.'); } const result = yield* files.uploadVersion({ id: params.id, diff --git a/apps/web/src/routes/api/internal/maintenance/+server.ts b/apps/web/src/routes/api/internal/maintenance/+server.ts index ee3b3d7..8ae3cc1 100644 --- a/apps/web/src/routes/api/internal/maintenance/+server.ts +++ b/apps/web/src/routes/api/internal/maintenance/+server.ts @@ -6,11 +6,12 @@ import { runAcrossOrgs, runEdgeWithEvent } from '$lib/server/edge'; import { Unauthorized } from '$lib/server/errors'; import { Lifecycle, summarize } from '$lib/server/services/lifecycle'; -// The cron tick: one global pass (device codes), then one bounded pass -// per randomly chosen live org so no tenant starves another. +// The cron tick: one global pass (device codes, trust promotions), then +// one bounded pass per randomly chosen live org so no tenant starves +// another. export const POST: RequestHandler = async (event) => { const { request } = event; - const global = await runEdgeWithEvent( + const { global, trust } = await runEdgeWithEvent( event, Effect.gen(function* () { const config = yield* AppConfig; @@ -31,7 +32,9 @@ export const POST: RequestHandler = async (event) => { message: 'Scheduled request is unauthorized' }); } - return yield* lifecycle.global; + const global = yield* lifecycle.global; + const trust = yield* lifecycle.trust; + return { global, trust }; }) ); if (!event.platform) return new Response(null, { status: 204 }); @@ -43,6 +46,7 @@ export const POST: RequestHandler = async (event) => { JSON.stringify({ message: 'maintenance tick', orgs: perOrg.length, + trustPromotions: trust, ...summarize( global, perOrg.map((entry) => entry.value) diff --git a/apps/web/src/routes/api/sites/sessions/+server.ts b/apps/web/src/routes/api/sites/sessions/+server.ts index 1927080..0a92208 100644 --- a/apps/web/src/routes/api/sites/sessions/+server.ts +++ b/apps/web/src/routes/api/sites/sessions/+server.ts @@ -4,10 +4,12 @@ import { } from '@adrive/shared'; import type { RequestHandler } from './$types'; import { Effect, Schema } from 'effect'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { runEdge } from '$lib/server/edge'; import { requireWrite } from '$lib/server/request-auth'; import { InvalidRequest } from '$lib/server/errors'; import { readBoundedJson } from '$lib/server/request-json'; +import { RateLimits } from '$lib/server/services/rate-limits'; import { Sites } from '$lib/server/services/sites'; const MAX_MANIFEST_BYTES = 1024 * 1024; @@ -37,7 +39,12 @@ export const POST: RequestHandler = (event) => { return runEdge( Effect.gen(function* () { const sites = yield* Sites; - yield* requireWrite(event); + const rateLimits = yield* RateLimits; + const credential = yield* requireWrite(event); + const rateLimit = yield* rateLimits.upload(credential.orgId); + if (!rateLimit.allowed) { + return rateLimitResponse('Too many uploads. Try again later.'); + } const input: SiteSessionCreate = yield* readManifest(request); return Response.json(yield* sites.createSession(input), { status: 201 }); }) diff --git a/apps/web/src/routes/f/[id]/+server.ts b/apps/web/src/routes/f/[id]/+server.ts index 007cb17..9f93725 100644 --- a/apps/web/src/routes/f/[id]/+server.ts +++ b/apps/web/src/routes/f/[id]/+server.ts @@ -17,11 +17,13 @@ import { import { AppConfig } from '$lib/server/config'; import { shouldRecordFileDownload } from '$lib/server/auth-policy'; import { decodeRangeHeader, rangeHeaders } from '$lib/server/download-response'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { runEdge } from '$lib/server/edge'; import { NotFound, StorageError } from '$lib/server/errors'; import { Blobs } from '$lib/server/services/blobs'; import { Files } from '$lib/server/services/files'; import { GrantSecrets } from '$lib/server/services/grant-secrets'; +import { RateLimits } from '$lib/server/services/rate-limits'; const requestedVersion = (url: URL) => { const value = url.searchParams.get('v'); @@ -30,7 +32,13 @@ const requestedVersion = (url: URL) => { return Number.isSafeInteger(version) && version > 0 ? version : null; }; -const serveFile: RequestHandler = ({ params, platform, request, url }) => +const serveFile: RequestHandler = ({ + getClientAddress, + params, + platform, + request, + url +}) => runEdge( Effect.gen(function* () { const config = yield* AppConfig; @@ -43,6 +51,11 @@ const serveFile: RequestHandler = ({ params, platform, request, url }) => if (thumbnailSource && !hasGrant) { return yield* new NotFound({ id: params.id }); } + // Invalid ids and denied grants cost database work too. + const rateLimits = yield* RateLimits; + const rateLimit = yield* rateLimits.anonymous(getClientAddress()); + if (!rateLimit.allowed) return rateLimitResponse(); + const content = yield* files.findContent(params.id, version, hasGrant); const privateResponse = hasGrant || !content.file.public; const dashboardPreview = diff --git a/apps/web/src/routes/report/+server.ts b/apps/web/src/routes/report/+server.ts new file mode 100644 index 0000000..8a7dbdd --- /dev/null +++ b/apps/web/src/routes/report/+server.ts @@ -0,0 +1,165 @@ +import type { RequestHandler } from './$types'; +import { Effect, Schema } from 'effect'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; +import { AppConfig } from '$lib/server/config'; +import { runEdge } from '$lib/server/edge'; +import { InvalidRequest, NotFound } from '$lib/server/errors'; +import { + REPORT_DETAILS_MAX, + REPORT_REASONS, + ReportCreateSchema, + hashReporterIp +} from '$lib/server/report-policy'; +import { readBoundedJson, readBoundedText } from '$lib/server/request-json'; +import { Admin } from '$lib/server/services/admin'; +import { RateLimits } from '$lib/server/services/rate-limits'; + +// Abuse reports, on the content host that serves the file: anyone can +// file one, nobody has to sign in. GET renders the one form the site 404 +// page and public file links point at; POST takes the form or JSON. + +const MAX_BODY_BYTES = 8 * 1024; + +const escapeHtml = (value: string) => + value + .replaceAll('&', '&') + .replaceAll('<', '<') + .replaceAll('>', '>') + .replaceAll('"', '"'); + +const page = (body: string, status = 200) => + new Response( + ` + + + + + +Report content + + +
    ${body}
    +`, + { + status, + headers: { + 'Content-Type': 'text/html; charset=utf-8', + 'Cache-Control': 'private, no-store', + 'Content-Security-Policy': + "default-src 'none'; style-src 'unsafe-inline'; form-action 'self'; base-uri 'none'; frame-ancestors 'none'", + 'Referrer-Policy': 'no-referrer', + 'X-Content-Type-Options': 'nosniff' + } + } + ); + +const form = (fileId: string) => { + const options = REPORT_REASONS.map( + (reason) => + `` + ).join(''); + return page(`

    Report content

    +
    + + + + +
    `); +}; + +const readForm = (request: Request) => + readBoundedText(request, { + maxBytes: MAX_BODY_BYTES, + invalidLengthMessage: 'Report is too large', + invalidTextMessage: 'Report form is invalid' + }).pipe( + Effect.map((text) => { + const params = new URLSearchParams(text); + return { + fileId: params.get('fileId') ?? '', + reason: params.get('reason') ?? '', + ...(params.get('details') ? { details: params.get('details') } : {}) + }; + }) + ); + +const readReport = (request: Request) => + Effect.gen(function* () { + const isForm = (request.headers.get('content-type') ?? '').includes( + 'application/x-www-form-urlencoded' + ); + const raw: unknown = isForm + ? yield* readForm(request) + : yield* readBoundedJson(request, { + maxBytes: MAX_BODY_BYTES, + invalidLengthMessage: 'Report is too large', + invalidJsonMessage: 'A JSON report is required' + }); + const decoded = yield* Schema.decodeUnknownEffect(ReportCreateSchema)( + raw + ).pipe( + Effect.mapError( + () => + new InvalidRequest({ + status: 400, + message: `A file id and a reason (${REPORT_REASONS.join(', ')}) are required` + }) + ) + ); + return { ...decoded, isForm }; + }); + +export const GET: RequestHandler = ({ url }) => { + const fileId = url.searchParams.get('f')?.trim() ?? ''; + if (!fileId || fileId.length > 128) { + return page( + '

    Report content

    Open this page from the file you want to report.

    ', + 404 + ); + } + return form(fileId); +}; + +export const POST: RequestHandler = (event) => + runEdge( + Effect.gen(function* () { + const admin = yield* Admin; + const config = yield* AppConfig; + const rateLimits = yield* RateLimits; + const org = event.locals.content; + if (!org) return yield* new NotFound({ id: 'report' }); + const ip = event.getClientAddress(); + const rateLimit = yield* rateLimits.anonymous(ip); + if (!rateLimit.allowed) return rateLimitResponse(); + const report = yield* readReport(event.request); + const reporterIpHash = yield* Effect.promise(() => + hashReporterIp(ip, config.maintenanceSecret) + ); + const id = yield* admin.fileReport({ + orgId: org.orgId, + fileId: report.fileId, + reason: report.reason, + details: report.details?.trim() || null, + reporterIpHash + }); + if (report.isForm) { + return page( + '

    Thank you

    The report was received and will be reviewed.

    ' + ); + } + return Response.json( + { id }, + { status: 201, headers: { 'Cache-Control': 'private, no-store' } } + ); + }) + ); diff --git a/apps/web/src/routes/s/[id]/[...path]/+server.ts b/apps/web/src/routes/s/[id]/[...path]/+server.ts index 7d489a9..e532835 100644 --- a/apps/web/src/routes/s/[id]/[...path]/+server.ts +++ b/apps/web/src/routes/s/[id]/[...path]/+server.ts @@ -9,6 +9,7 @@ import { siteCacheControl } from '$lib/server/content-cache'; import { AppConfig } from '$lib/server/config'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { runEdge } from '$lib/server/edge'; import { NotFound } from '$lib/server/errors'; import { isSiteVersionRequestServable } from '$lib/server/site-policy'; @@ -16,6 +17,7 @@ import { Blobs } from '$lib/server/services/blobs'; import { Files } from '$lib/server/services/files'; import { Sites } from '$lib/server/services/sites'; import { GrantSecrets } from '$lib/server/services/grant-secrets'; +import { RateLimits } from '$lib/server/services/rate-limits'; export const trailingSlash = 'ignore'; @@ -42,7 +44,12 @@ const siteGrant = (path: string) => { }; }; -const serveSite: RequestHandler = ({ params, request, url }) => +const serveSite: RequestHandler = ({ + getClientAddress, + params, + request, + url +}) => runEdge( Effect.gen(function* () { const grant = siteGrant(params.path ?? ''); @@ -62,6 +69,12 @@ const serveSite: RequestHandler = ({ params, request, url }) => const blobs = yield* Blobs; const files = yield* Files; const config = yield* AppConfig; + const rateLimits = yield* RateLimits; + // Site assets are not held in the edge cache by the Worker (the CDN + // honours their Cache-Control), so every request that reaches here + // is a cache miss and counts. + const rateLimit = yield* rateLimits.anonymous(getClientAddress()); + if (!rateLimit.allowed) return rateLimitResponse(); // The asset row names the owning org, which the grant is bound to, // so it is resolved before the signature is checked. const asset = yield* sites.findAsset( diff --git a/apps/web/src/routes/t/[id]/[version]/grid.webp/+server.ts b/apps/web/src/routes/t/[id]/[version]/grid.webp/+server.ts index bcf1039..9aa332b 100644 --- a/apps/web/src/routes/t/[id]/[version]/grid.webp/+server.ts +++ b/apps/web/src/routes/t/[id]/[version]/grid.webp/+server.ts @@ -20,12 +20,14 @@ import { storeEdgeCache, thumbnailCacheControl } from '$lib/server/content-cache'; +import { rateLimitResponse } from '$lib/server/auth-rate-limit-response'; import { runEdge } from '$lib/server/edge'; import { NotFound, StorageError } from '$lib/server/errors'; import { Blobs } from '$lib/server/services/blobs'; import { currentContentOrigin } from '$lib/server/services/current-org'; import { Files } from '$lib/server/services/files'; import { GrantSecrets } from '$lib/server/services/grant-secrets'; +import { RateLimits } from '$lib/server/services/rate-limits'; const parsedVersion = (value: string) => { const version = Number(value); @@ -66,7 +68,13 @@ const publicThumbnailRedirect = (url: URL) => { }); }; -export const GET: RequestHandler = ({ params, platform, request, url }) => +export const GET: RequestHandler = ({ + getClientAddress, + params, + platform, + request, + url +}) => runEdge( Effect.gen(function* () { const version = parsedVersion(params.version); @@ -136,6 +144,10 @@ export const GET: RequestHandler = ({ params, platform, request, url }) => } } + const rateLimits = yield* RateLimits; + const rateLimit = yield* rateLimits.anonymous(getClientAddress()); + if (!rateLimit.allowed) return rateLimitResponse(); + const blobs = yield* Blobs; const cached = content.thumbnailR2Key === null diff --git a/apps/web/worker-configuration.d.ts b/apps/web/worker-configuration.d.ts index aa6ff40..ffb627b 100644 --- a/apps/web/worker-configuration.d.ts +++ b/apps/web/worker-configuration.d.ts @@ -1,11 +1,15 @@ /* eslint-disable */ -// Generated by Wrangler by running `wrangler types` (hash: 6c1287863c727955d614dd4a3d807365) +// Generated by Wrangler by running `wrangler types` (hash: d76966b5dbd8efc44b28675760946f8e) // Runtime types generated with workerd@1.20260722.1 2026-07-27 global_fetch_strictly_public,nodejs_compat interface __BaseEnv_Env { AUTH_GUARD: KVNamespace; BUCKET: R2Bucket; HYPERDRIVE: Hyperdrive; JOBS: Queue; + RL_UPLOAD: RateLimit; + RL_PUBLISH: RateLimit; + RL_AUTH: RateLimit; + RL_ANON: RateLimit; BROWSER: BrowserRun; AI?: Ai; ASSETS: Fetcher; @@ -29,6 +33,10 @@ declare namespace Cloudflare { BUCKET: R2Bucket; HYPERDRIVE: Hyperdrive; JOBS: Queue; + RL_UPLOAD: RateLimit; + RL_PUBLISH: RateLimit; + RL_AUTH: RateLimit; + RL_ANON: RateLimit; BROWSER: BrowserRun; AI: Ai; ASSETS: Fetcher; diff --git a/apps/web/wrangler.jsonc b/apps/web/wrangler.jsonc index 69ce047..cecd9a3 100644 --- a/apps/web/wrangler.jsonc +++ b/apps/web/wrangler.jsonc @@ -47,6 +47,31 @@ "id": "419a52adcd9649e0a24e5c47c2748477" } ], + // Abuse brakes (services/rate-limits.ts). Per-colo and approximate: + // fine for stopping a runaway client, not for billing. + "ratelimits": [ + { + "name": "RL_UPLOAD", + "namespace_id": "1001", + "simple": { "limit": 60, "period": 60 } + }, + { + "name": "RL_PUBLISH", + "namespace_id": "1002", + "simple": { "limit": 10, "period": 60 } + }, + { + "name": "RL_AUTH", + "namespace_id": "1003", + // Shared creation + polling budget; a device polls 12 times/minute. + "simple": { "limit": 30, "period": 60 } + }, + { + "name": "RL_ANON", + "namespace_id": "1004", + "simple": { "limit": 300, "period": 60 } + } + ], "queues": { "producers": [{ "queue": "adrive-jobs", "binding": "JOBS" }], "consumers": [ @@ -96,8 +121,9 @@ // `*.files.davis7.space` needs explicit certificate coverage on the // davis7.space zone, or use a dedicated content-domain zone apex. // 5. `wrangler secret put` MAINTENANCE_SECRET, WORKOS_API_KEY, - // WORKOS_CLIENT_ID, WORKOS_COOKIE_PASSWORD, WORKOS_WEBHOOK_SECRET - // (all `--env production`) + // WORKOS_CLIENT_ID, WORKOS_COOKIE_PASSWORD, WORKOS_WEBHOOK_SECRET, + // ADMIN_USER_IDS; optionally URLSCAN_API_KEY + CF_ACCOUNT_ID and + // CF_API_TOKEN + CF_ZONE_ID (docs/abuse.md) (all `--env production`) // 6. `DATABASE_URL=postgres://... bun db:pg:migrate` (see docs/release.md) "env": { "production": { @@ -155,6 +181,28 @@ "id": "09b884a95e5e40389d8554b3cd690d3f" } ], + "ratelimits": [ + { + "name": "RL_UPLOAD", + "namespace_id": "2001", + "simple": { "limit": 60, "period": 60 } + }, + { + "name": "RL_PUBLISH", + "namespace_id": "2002", + "simple": { "limit": 10, "period": 60 } + }, + { + "name": "RL_AUTH", + "namespace_id": "2003", + "simple": { "limit": 30, "period": 60 } + }, + { + "name": "RL_ANON", + "namespace_id": "2004", + "simple": { "limit": 300, "period": 60 } + } + ], "queues": { "producers": [{ "queue": "adrive-jobs-production", "binding": "JOBS" }], "consumers": [ diff --git a/bun.lock b/bun.lock index d3527f1..557448c 100644 --- a/bun.lock +++ b/bun.lock @@ -24,6 +24,7 @@ "effect": "4.0.0-beta.102", "markdown-it": "^14.3.0", "mime": "^4.1.0", + "parse5": "^8.0.1", "pg": "^8.22.0", "runed": "^0.37.1", "zod": "^4.4.3", @@ -778,6 +779,8 @@ "once": ["once@1.4.0", "", { "dependencies": { "wrappy": "1" } }, "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w=="], + "parse5": ["parse5@8.0.1", "", { "dependencies": { "entities": "^8.0.0" } }, "sha512-z1e/HMG90obSGeidlli3hj7cbocou0/wa5HacvI3ASx34PecNjNQeaHNo5WIZpWofN9kgkqV1q5YvXe3F0FoPw=="], + "parseurl": ["parseurl@1.3.3", "", {}, "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ=="], "partyserver": ["partyserver@0.5.10", "", { "dependencies": { "nanoid": "^5.1.9" }, "peerDependencies": { "@cloudflare/workers-types": "^4.20260424.1 || ^5.20260703.1" } }, "sha512-t2B3mhTL1IOxCsj8rZgn4TxTuub7oLhypjc1/fGPNsbgnn9OsHms6uR3QEd5bFJ/7xrFo771j3DdUlll8cxgBg=="], @@ -1026,6 +1029,8 @@ "negotiator/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], + "parse5/entities": ["entities@8.1.0", "", {}, "sha512-kxL7msIffSuh9aaFAMD7rxAIuTRMAHMeBtgHW2yUdWw732ZNh4MehkF2gdjvtdmikkaIP9bFDDJOPlsvm7avrA=="], + "postcss/nanoid": ["nanoid@3.3.19", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-Y2tUNy4ouw6tq5oDSKeQYGOyhkUBhNOcGV/02KC+6kd9eDGqdZd++mjMiIDilrBYvjEnCYvVtsuHCuP+okSfug=="], "router/path-to-regexp": ["path-to-regexp@8.4.2", "", {}, "sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA=="], diff --git a/docs/abuse.md b/docs/abuse.md new file mode 100644 index 0000000..73b3a8d --- /dev/null +++ b/docs/abuse.md @@ -0,0 +1,207 @@ +# Abuse controls + +One bad tenant must not take the content domain down, and an operator +must be able to stop one in under a minute. This is what the code does, +what the Cloudflare zone must be set to by hand, and how to run the +controls. + +## Zone checklist (not code) + +Do these once on the zone that owns `CONTENT_DOMAIN`, and again whenever +the content domain moves. Tenant content is only ever served from +`.`; the dashboard origin is a separate host. + +- **WAF managed rules** on the content zone: enable the Cloudflare + Managed Ruleset and the OWASP Core Ruleset in blocking mode. Content + responses are static files, so false positives are rare; if one appears + add a skip rule scoped to `/f/*`, never to the whole zone. +- **Bot Fight Mode** (or Super Bot Fight Mode on a paid plan): on. Scraper + traffic against tenant hosts otherwise counts against the per-IP rate + limit and R2 egress. +- **Hotlink protection: off.** Serving files from elsewhere is the product. +- **Rate limiting rules** at the zone level are a second layer over the + Worker bindings below; a rule of the shape "more than 1000 requests per + minute from one IP to `*./*` → managed challenge" is + enough. +- **Cache**: leave the default cache level. The Worker sets `Cache-Control` + per file (immutable for pinned versions, revalidate for sites) and + purges by URL and by host when a verdict changes (below), so no zone + cache rule is needed. +- **Netcraft**: register the content domain as a hosting provider in the + Netcraft takedown feed so phishing reports arrive at `abuse@` instead + of the registrar. +- **DMCA agent**: register a designated agent with the US Copyright Office + for the operating entity and publish the contact on the landing site. +- **abuse@ mailbox**: create it, route it to the same people who watch + `/admin`, and list it on the landing site next to the DMCA agent. +- **Reports**: every tenant host answers `GET /report?f=` with a + form; link it from the landing site's abuse page so a reporter who + lands on the bare domain finds it. + +## Trust levels + +`orgs.trust` (`apps/web/src/lib/server/trust-policy.ts`): + +| Level | Reached by | May publish | Scan | +| ------------- | ----------------------------------------------------------------------------- | --------------- | ---------------- | +| `new` | Sign-up | No (403) | n/a | +| `verified` | A sign-in with a verified email | Yes | Before it's live | +| `established` | Org over 14 days old and currently paid (maintenance sweep), or an admin bump | Yes | After it's live | +| `suspended` | The kill switch | No; host is 404 | n/a | + +"Publish" is anything that makes content public: an upload that lands +public (including HTML, which is always public), a visibility change, a +rename to `.html`, a site session or commit. A `new` org gets 403 "Verify +your email to share publicly". The free plan can share publicly once +verified; every publish by a verified org is scanned first. + +Bumping trust is an admin action on `/admin` (or `PATCH +/api/admin/orgs/` with `{ "action": "trust", "trust": "established" }`). + +## Scan pipeline + +`apps/web/src/lib/server/services/scanner.ts`. Every version that becomes +public gets a `scan` job on the queue. For a verified org the row stays +private with `files.publish_pending = true` until the scan clears it; for +an established org it is public at once and scanned after. + +Each version write records its scan obligation in Postgres before enqueueing. +The lifecycle sweep requeues overdue obligations in bounded batches, so a queue +outage cannot silently lose the initial scan. A completed suspicious verdict +ends automatic scanning and leaves the file for review. When only some URL +submissions succeed, their results are still collected; failed submissions +keep a suspicious minimum verdict, and any malicious result quarantines. + +Publishing the current version does not expose unscanned older bytes. Older +versions need their own completed clean scan or operator clearance for anonymous +access. Owners can still request signed links for pending/private versions; +a malicious version stays blocked even with an existing grant. Clearing the +current version does not clear older versions. Admin actions include the +displayed current version and reject an outdated overview with a +refresh-required response. + +Checks, in order, each writing one `scan_verdicts` row per +`(file, version, source)`: + +1. `hash`: the object's SHA-256 (objects up to 32 MiB; larger ones are + recorded as skipped) against `blocked_hashes`. A hit is `malicious`. + The hash is also stored on `file_versions.sha256`. +2. `sniff`: the first 512 bytes against the declared content type + (`mime-sniff.ts`). HTML, SVG, PE/ELF/Mach-O, or a shell script served + under a type that does not admit it is `suspicious`. +3. `urlscan`: for HTML files and sites, `
    `, `