Repository navigation
Enterprise: custom plans, time-boxed operator activation and out-of-catalogue billing #81
Description
Context
Onboarding the first large prospect (agent.wally BV, ~66B tokens/month, migrating from Vertex AI / Sonnet 5) required provisioning a time-boxed Scale trial by hand, directly against the production database. That exercise surfaced three structural gaps. Every one of them will recur on the next deal, and each currently forces manual SQL on a customer's org.
Gap 1 — No per-org token cap override
config.CapForTier(org.Tier, model) reads a hardcoded map in internal/config/tier_caps.go. There is no per-organization override anywhere, so a custom allowance (e.g. 50B of glm5.3 against Scale's published 18B) cannot be expressed at all.
Notes for whoever picks this up:
- Adding a tier key is the smaller change, but
TierImageQuotafalls back todefaultImageQuota(300) for an unknown tier, andTierKeyLimitswould need a matching row. sota_caps_invariant_test.gorequires every tier to carry a row for every model inSotaModels.credits.FlatForis deliberately fail-open on an unknown (tier, model) pair, so a missing row grants UNLIMITED flat coverage rather than refusing. Any new tier must be added to both maps in the same commit.
Gap 2 — No time-boxed operator activation
There is no endpoint to activate an organisation that already exists. POST /api/admin/organizations creates a new one; the only activation path is checkout.session.completed.
A Stripe trial is NOT a workaround, and this is worth recording because it looks like one:
organizations_subscription_status_checkallows onlypending, active, past_due, canceled, payment_pending, payment_failed, unsubscribed. Stripe sendstrialing, which violates the CHECK and would put the webhook into a 500 loop.AccessLevelForStatus("trialing")returnsAccessBillingOnly(pinned byinternal/middleware/subscription_test.go:65), the most restrictive level. A trial would grant strictly less access thanunsubscribed.onSubscriptionCreatedonly acts when the org's stored subscription id is non-empty and different from the incoming one ("the unambiguous signature of a returning customer"), so creating a subscription out of band is a no-op.
The workaround used today is a 100%-off coupon on a real Checkout Session, then cancelling the subscription on the agreed date. It works, but it leaves a caveat: customer.subscription.deleted marks the org canceled while keys keep working until current_period_end, which is a month out. The trial therefore does not actually stop on its own and the key must be revoked manually.
Wanted: a first-class trial/grant state with an explicit expiry that revokes access at that instant.
Gap 3 — No out-of-catalogue billing
The agreed commercial shape is Scale (EUR 3,199) plus EUR 4,600 for a GLM 5.3 volume block, with further blocks at the same unit rate. Nothing models this: tiers map 1:1 to a Stripe price, and the SOTA add-on is per-key at a fixed EUR 150, with the monthly window fixed per tier.
Also worth fixing while in here
keyRPMLimit/keyMaxParallelReqs/keyTPMLimitare package constants ininternal/handlers/keys.go. Seat consolidation (migration 055,slot_cost) scales them, but there is still no way to grant an org limits beyond its seat entitlement.- A consolidated key concentrates many seats behind one credential, which is the exact signature the key-sharing detector looks for (
platform/devops/API-KEY-SHARING-DETECTION.md). Consolidated keys need an exemption or the detector will flag our own customers.
Acceptance
- An operator can grant an organisation a custom allowance, custom key limits and an expiry, without writing SQL.
- Expiry actually cuts access at the stated time.
- The commercial shape above is representable end to end, from quote to invoice.
Activity
sre-helmcode commentedon Sep 15, 2026
Two more from the agent.wally onboarding on 2026-09-15, both small and both in the same family as the gaps above.
A 502 with a clear cause renders as "Failed to fetch". Enabling the GLM 5.3 add-on from the console failed with a generic network error. The API had answered 502 with the real reason in the log:
The backend behaved correctly: it declined to grant a EUR 150/key add-on it could not bill. But nothing in the UI said "there is no payment method", so the operator's first read was "our product is broken". Surface the API's reason on that call, or at minimum distinguish a 502 with a body from a transport failure.
Consolidated keys will be flagged by our own key-sharing detector. Seat consolidation (migration 055,
slot_cost) is now sold as a feature: one key carrying 30 seats at 3000 rpm / 150 concurrent. Many source IPs behind one credential with high fan-out is precisely the signatureplatform/devops/API-KEY-SHARING-DETECTION.mdlooks for. Either exempt keys withslot_cost > 1, or scale the detector's thresholds byslot_cost. Without it we will open abuse cases against the customers who bought the biggest plans.Related: #82 covers the enterprise edge allowlist, which came out of the same session.
sre-helmcode commentedon Sep 16, 2026
Correction to the second item in the comment above: the key-sharing detector cannot see a consolidated key
I wrote that "consolidated keys will be flagged by our own key-sharing detector" and filed it as something to fix before the next deal. Measured 2026-09-16, that is not true today, and the reason is worth recording so nobody spends a sprint on it, and so we know exactly what would make it true.
The detector has one source, and it is the community database:
docker/abuse-detection-etl/etl.pyreadsLiteLLM_SpendLogsfrom that DSN and keeps a single watermark row,source='litellm_community'. There is no enterprise branch, no second DSN and no second watermark.slot_costis an enterprise-only column (055_api_keys_slot_cost,enterprise-api). It does not exist incloud-apiat all -- community keys are one per member at 100 rpm / 5 parallel, so nothing there can be consolidated in the first place.So the two halves never meet: the keys that carry seats live in
litellm-enterprise, and the detector only ever readslitellm-community. agent.wally's 30-seat key is invisible to it.The trigger that would make this live, stated so it is checkable rather than folklore: pointing the ETL at the enterprise LiteLLM database -- a second
LITELLM_DSN, a secondetl_watermarksource, or merging the two LiteLLM databases. On the day that happens, the exemption has to land in the same change, because a 30-seat key at 3000 rpm / 150 concurrent from many offices is the exact S1+S6 signature the scorer is built to find, and the customers it would open cases against are the ones who bought the biggest plans.The fix at that point is still the one in the comment: exempt
slot_cost > 1, or scaleMIN_CLUSTER_REQS/RED_CONCURRENCYbyslot_cost. Scaling is the better half of the two -- an exemption makes a consolidated key a blind spot for real abuse, which is the opposite of what we want on our largest accounts.The first item in that comment (a 502 with a real reason rendering as "Failed to fetch") stands unchanged and is live.