Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,8 @@ When an account is refused, it rests until the provider's own reset (its `retry-

A reserve, set per account on the Accounts page or with `PATCH /admin/api/accounts/{id}` and `{"reserve_percent": 5}`, is a hard floor on the account's longest window (monthly over weekly over daily over five-hour): once that window reaches the floor, douane stops using the account, live conversations included, until it resets. To watch other windows, pick them on the Accounts page or send `"reserve_windows": ["five-hour", "weekly"]` (any of `five-hour`, `daily`, `weekly`, `monthly`; `[]` returns to the longest window): the account then stops as soon as one of them reaches the floor. Close to the floor, quotas are reread at most once a minute, so a burst can overshoot by a little.

What is left of a week is lost at its reset, so the last hours before an account's weekly reset lift its limits: the reserve and client shares stop applying and the account serves until it runs out. The default is 6 hours; change it in Settings > End of the week or with `free_before_reset_hours` (0 to 168, 0 keeps the limits to the end). An account whose longest window is a month uses its monthly reset instead; accounts with only shorter windows are never lifted.

Banked resets for Claude and Codex appear on each account card. Using one asks for confirmation, and a retried click reuses the same request id, so a double click cannot spend two.

![Overview page: active accounts, requests, tokens, cache reads and recent activity](docs/images/activity.png)
Expand All @@ -227,7 +229,7 @@ Banked resets for Claude and Codex appear on each account card. Using one asks f
With no client defined, anyone who reaches the client API can use every account. Create a client on the **Clients** page, with `POST /admin/api/clients` or `douanectl client add`, and from then on every `/v1` call needs a client key, sent as the tool's API key (`Authorization: Bearer dk_...` or `x-api-key`). Other keys get 401. The key is shown once; douane keeps only its hash. Deleting the last client opens the API again.

- **Access.** A client reaches every account, accounts connected later included, or only the providers, custom routes and single accounts you pick. `/v1/models` and `/v1/quotas` list only what it may use.
- **Shares.** A share is a percentage of each account-wide window (five hours, day, week, month) of every account of a provider. Shares of a provider add up to 100 % at most. What no share covers is common to every client. A client's unused share stays reserved for it until the last seventh of the window (the last day of a week), then any client may use it. A client with no share uses the common part, then freed shares. A client held back gets 429 until the window resets.
- **Shares.** A share is a percentage of each account-wide window (five hours, day, week, month) of every account of a provider. Shares of a provider add up to 100 % at most. What no share covers is common to every client. A client's unused share stays reserved for it until the last seventh of the window (the last day of a week), then any client may use it. A client with no share uses the common part, then freed shares. A client held back gets 429 until the window resets. In the hours before the weekly reset (`free_before_reset_hours`, see above), shares hold no one back.
- **Usage.** Providers report usage per account, not per caller, so douane attributes it: each rise of a window goes to the clients whose calls ended on that account since the previous reading, weighted by tokens, or to the calls still running. A rise that no call through douane explains counts as outside use. Calls of several clients interleaving on one account make the split an estimate; one client per account keeps it exact.
- **Quota bars.** With a client key, `/v1/quotas` shows each window as the part the client has used of what it may use, so a full bar means it is held back.

Expand Down
85 changes: 76 additions & 9 deletions crates/douane/src/clients.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
//! account-wide window (five hours, day, week, month) of every account in a
//! pool. Unused shares stay reserved for their holders until the last seventh
//! of the window, when anyone may use them; what no share covers is common.
//! In the hours before an account's weekly reset (`free_before_reset_hours`),
//! shares hold no one back.
//!
//! Usage is attributed, not read: each rise of a window's utilization goes to
//! the clients whose calls ended on that account since the previous reading,
Expand Down Expand Up @@ -679,16 +681,19 @@ impl Clients {

/// The reset of the window where `client` has reached what the other
/// holders' shares leave it on this account; `None` when it may use it.
/// Within `free_before` seconds of the account's weekly reset, shares
/// no longer hold anyone back.
pub fn share_blocks(
&self,
client: Option<&Client>,
account: &AccountCredential,
snapshot: Option<&QuotaSnapshot>,
now: i64,
free_before: i64,
) -> Option<i64> {
let snapshot = snapshot?;
let holders = self.holders(account.pool());
if holders.is_empty() {
if holders.is_empty() || snapshot.longest_window_ends_within(free_before, now) {
return None;
}
let me = client.map(|c| c.id.as_str()).unwrap_or("");
Expand All @@ -704,16 +709,18 @@ impl Clients {

/// The snapshot as `client` sees it: each account-wide window shows the
/// part of what it may use that it has used, so a bar at 100% means it
/// is held back. Windows of pools without shares stay as they are.
/// is held back. Windows of pools without shares, or of an account whose
/// shares are freed before its weekly reset, stay as they are.
pub fn client_view(
&self,
client: &Client,
mut snapshot: QuotaSnapshot,
pool: &str,
now: i64,
free_before: i64,
) -> QuotaSnapshot {
let holders = self.holders(pool);
if holders.is_empty() {
if holders.is_empty() || snapshot.longest_window_ends_within(free_before, now) {
return snapshot;
}
let account = snapshot.account_id.clone();
Expand Down Expand Up @@ -966,42 +973,102 @@ mod tests {
clients.observe(&snapshot("a", 70.0, reset), NOW);
// Perso may still use 5 points before touching famille's 25.
assert_eq!(
clients.share_blocks(Some(&perso), &a, Some(&snapshot("a", 70.0, reset)), NOW),
clients.share_blocks(Some(&perso), &a, Some(&snapshot("a", 70.0, reset)), NOW, 0),
None
);
clients.call_ended("a", Some(&perso.id), 1.0);
clients.observe(&snapshot("a", 75.0, reset), NOW);
let full = snapshot("a", 75.0, reset);
assert_eq!(
clients.share_blocks(Some(&perso), &a, Some(&full), NOW),
clients.share_blocks(Some(&perso), &a, Some(&full), NOW, 0),
Some(reset)
);
// Famille keeps its share.
assert_eq!(
clients.share_blocks(Some(&famille), &a, Some(&full), NOW),
clients.share_blocks(Some(&famille), &a, Some(&full), NOW, 0),
None
);
// Perso's bar is full; famille's shows nothing used of its 25.
let view = |c: &Client| {
clients.client_view(c, full.clone(), "claude", NOW).windows[0].remaining_percent
clients
.client_view(c, full.clone(), "claude", NOW, 0)
.windows[0]
.remaining_percent
};
assert_eq!(view(&perso), Some(0.0));
assert_eq!(view(&famille), Some(100.0));
// In the last day of the week, famille's unused share opens to all.
let late = NOW + 3 * 86_400 - 20 * 3600;
assert_eq!(
clients.share_blocks(Some(&perso), &a, Some(&full), late),
clients.share_blocks(Some(&perso), &a, Some(&full), late, 0),
None
);
// Accounts of a pool without shares never hold anyone back.
let mut codex = account("c");
codex.provider = SubscriptionProvider::Codex;
assert_eq!(
clients.share_blocks(Some(&perso), &codex, Some(&full), NOW),
clients.share_blocks(Some(&perso), &codex, Some(&full), NOW, 0),
None
);
}

#[test]
fn shares_stop_holding_back_hours_before_the_weekly_reset() {
let (_dir, clients) = clients();
let (_famille, _) = clients
.create(ClientPatch {
name: Some("famille".into()),
shares: Some([("claude".to_owned(), 25.0)].into()),
..ClientPatch::default()
})
.unwrap();
let (perso, _) = clients.create(named("perso")).unwrap();
let a = account("a");
// The week ends in five hours; a five-hour window started an hour ago.
let weekly = NOW + 5 * 3600;
let five = NOW + 4 * 3600;
let with = |used: f64| {
let mut snapshot = snapshot("a", 10.0, weekly);
let mut window = snapshot.windows[0].clone();
window.id = "five-hour".into();
window.remaining_percent = Some(100.0 - used);
window.reset_at = Some(
time::OffsetDateTime::from_unix_timestamp(five)
.unwrap()
.format(&time::format_description::well_known::Rfc3339)
.unwrap(),
);
snapshot.windows.push(window);
snapshot
};
clients.observe(&with(0.0), NOW);
clients.call_ended("a", Some(&perso.id), 1.0);
clients.observe(&with(75.0), NOW);
let full = with(75.0);
// Kept to the end, famille's five-hour share holds perso back.
assert_eq!(
clients.share_blocks(Some(&perso), &a, Some(&full), NOW, 0),
Some(five)
);
assert_eq!(
clients.share_blocks(Some(&perso), &a, Some(&full), NOW, 4 * 3600),
Some(five)
);
// Six hours before the weekly reset, nothing is held back any more.
assert_eq!(
clients.share_blocks(Some(&perso), &a, Some(&full), NOW, 6 * 3600),
None
);
let view = |free_before: i64| {
clients
.client_view(&perso, full.clone(), "claude", NOW, free_before)
.windows[1]
.remaining_percent
};
assert_eq!(view(0), Some(0.0));
assert_eq!(view(6 * 3600), Some(25.0));
}

#[test]
fn usage_survives_a_restart() {
let (dir, clients) = clients();
Expand Down
9 changes: 8 additions & 1 deletion crates/douane/src/management.rs
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ async fn account_value(
&summary.reserve_windows,
snapshot.as_ref(),
now,
state.free_before_reset_seconds().await,
);
let mut value = json!(summary);
value["reserve_measured"] = json!(measured);
Expand Down Expand Up @@ -852,7 +853,13 @@ pub async fn public_quotas(
.route
.as_deref()
.unwrap_or(account.provider.as_str());
Some(state.clients.client_view(client, snapshot, pool, now))
Some(state.clients.client_view(
client,
snapshot,
pool,
now,
settings.free_before_reset_seconds(),
))
})
.collect();
}
Expand Down
12 changes: 12 additions & 0 deletions crates/douane/src/settings.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ pub struct Settings {
pub quota_display: BTreeMap<String, QuotaDisplay>,
/// Which windows `/v1/quotas` shows for each provider, and the one shown first.
pub quota_windows: BTreeMap<String, QuotaWindows>,
/// Hours before an account's weekly reset (monthly when its longest
/// window is a month) when its reserve and client shares stop applying,
/// so what is left gets used. 0 keeps them to the end.
pub free_before_reset_hours: u32,
}

#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
Expand Down Expand Up @@ -89,6 +93,7 @@ impl Default for Settings {
default_provider: None,
quota_display: BTreeMap::new(),
quota_windows: BTreeMap::new(),
free_before_reset_hours: 6,
}
}
}
Expand Down Expand Up @@ -151,6 +156,9 @@ impl Settings {
if !(10..=1800).contains(&self.request_timeout_seconds) {
bail!("request_timeout_seconds must be between 10 and 1800");
}
if self.free_before_reset_hours > 168 {
bail!("free_before_reset_hours must be between 0 and 168");
}
if let Some(p) = &self.default_provider {
if crate::subscriptions::SubscriptionProvider::parse(p).is_none() {
bail!("unknown default provider");
Expand Down Expand Up @@ -178,6 +186,10 @@ impl Settings {
Ok(())
}

pub fn free_before_reset_seconds(&self) -> i64 {
i64::from(self.free_before_reset_hours) * 3600
}

pub fn quota_windows(
&self,
provider: crate::subscriptions::SubscriptionProvider,
Expand Down
Loading