Skip to content

getting-started.md

Latest commit

 

History

History
315 lines (235 loc) · 7.5 KB

getting-started.md

File metadata and controls

315 lines (235 loc) · 7.5 KB
Raw

Getting Started with @spooled/sdk

This guide walks you through installing, configuring, and using the Spooled SDK for the first time.

Prerequisites

Installation

npm install @spooled/sdk

Or with yarn/pnpm:

yarn add @spooled/sdk
pnpm add @spooled/sdk

Creating a Client

import { SpooledClient } from "@spooled/sdk";

const client = new SpooledClient({
  apiKey: "sp_live_your_api_key",
});

For local development or testing:

const client = new SpooledClient({
  apiKey: process.env.SPOOLED_API_KEY,
  baseUrl: process.env.SPOOLED_API_URL || "https://api.spooled.cloud",
});

Your First Job

Creating a Job

const { id, created } = await client.jobs.create({
  queueName: "my-first-queue",
  payload: {
    message: "Hello, Spooled!",
    timestamp: Date.now(),
  },
});

console.log(`Created job: ${id}`);

Checking Job Status

const job = await client.jobs.get(id);

console.log(`Status: ${job.status}`);
// 'pending' | 'processing' | 'completed' | 'failed' | 'cancelled'

Listing Jobs

const jobs = await client.jobs.list({
  queueName: "my-first-queue",
  status: "pending",
  limit: 10,
});

for (const job of jobs) {
  console.log(`${job.id}: ${job.status}`);
}

Processing Jobs with a Worker

The SDK includes a built-in worker runtime for processing jobs:

import { SpooledClient, SpooledWorker } from "@spooled/sdk";

const client = new SpooledClient({ apiKey: "sp_live_..." });

const worker = new SpooledWorker(client, {
  queueName: "my-first-queue",
  concurrency: 5,
});

// Define how to process each job
worker.process(async (ctx) => {
  console.log(`Processing job ${ctx.jobId}`);
  console.log("Payload:", ctx.payload);

  // Your business logic here
  await doSomething(ctx.payload);

  // Optionally return a result
  return { success: true };
});

// Start the worker
await worker.start();

// Graceful shutdown on SIGTERM
process.on("SIGTERM", () => worker.stop());

Key Concepts

Queues

Queues are logical groupings of jobs. Jobs are processed in priority order within each queue.

// List all queues
const queues = await client.queues.list();

// Get queue statistics
const stats = await client.queues.getStats("my-queue");
console.log(`Pending: ${stats.pending}, Processing: ${stats.processing}`);

Priority

Jobs have a priority from -100 (lowest) to 100 (highest). Default is 0.

// High priority job
await client.jobs.create({
  queueName: "urgent",
  payload: { alert: "critical" },
  priority: 100,
});

// Low priority background task
await client.jobs.create({
  queueName: "background",
  payload: { cleanup: true },
  priority: -50,
});

Retries

Failed jobs are automatically retried based on the maxRetries setting:

await client.jobs.create({
  queueName: "emails",
  payload: { to: "user@example.com" },
  maxRetries: 5, // Retry up to 5 times
  timeoutSeconds: 60, // 60 second timeout per attempt
});

Scheduled Jobs

Delay job execution or run at a specific time:

// Execute in 5 minutes
await client.jobs.create({
  queueName: "notifications",
  payload: { reminder: true },
  scheduledAt: new Date(Date.now() + 5 * 60 * 1000),
});

Idempotency

Prevent duplicate jobs with idempotency keys:

const result = await client.jobs.create({
  queueName: "payments",
  payload: { orderId: "order-123" },
  idempotencyKey: "payment-order-123",
});

console.log(result.created); // false if job already exists

Plan Limits

All operations automatically enforce tier-based limits:

import { RateLimitError } from "@spooled/sdk";

try {
  await client.jobs.create({
    /* ... */
  });
} catch (error) {
  if (error instanceof RateLimitError && error.code === "QUOTA_EXCEEDED") {
    console.log(`Plan quota exceeded: ${error.message}`);
    console.log(`Request ID: ${error.requestId ?? "not provided"}`);
    // Only server metadata nested under `details` is preserved here.
    if (error.details) console.log("Details:", error.details);
  }
}

The SDK reliably preserves the HTTP status, code, message, request ID, and rate-limit headers. error.details is available only when the server nests metadata under details; top-level quota payload fields such as resource, current, limit, and plan are not currently exposed by the SDK.

Tier Active Jobs Daily Jobs Queues Workers
Free 10 1,000 5 3
Starter 100 100,000 25 25
Enterprise Unlimited Unlimited Unlimited Unlimited

Limits are enforced on:

  • ✅ Job creation (HTTP & gRPC)
  • ✅ Workflow creation
  • ✅ Schedule triggers
  • ✅ DLQ retry operations
  • ✅ Worker registration

Organization Management

Track usage and manage your organization:

// Get current usage and limits
const usage = await client.organizations.getUsage();
console.log(
  `Active jobs: ${usage.active_jobs.current}/${usage.active_jobs.limit}`,
);
console.log(`Plan: ${usage.plan.tier}`);

// Check if at risk of hitting limits
if (usage.active_jobs.percentage && usage.active_jobs.percentage > 80) {
  console.warn("Approaching active jobs limit!");
}

Dead Letter Queue

Manage failed jobs that have exhausted retries:

// List jobs in DLQ
const dlqJobs = await client.jobs.dlq.list({ limit: 100 });

// Retry failed jobs
await client.jobs.dlq.retry({
  queueName: "emails",
  limit: 50,
});

// Purge old failures
await client.jobs.dlq.purge({
  queueName: "emails",
  confirm: true,
  olderThan: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000), // 7 days old
});

Webhooks

Get notified when job events occur:

// Create webhook for job events
const webhook = await client.webhooks.create({
  name: "App notifications",
  url: "https://your-app.com/webhooks/spooled",
  events: ["job.completed", "job.failed"],
  secret: "webhook_secret_key",
});

// Retry a failed delivery
await client.webhooks.retryDelivery(webhookId, deliveryId);

Keep an eye on failures: after 20 consecutive failed deliveries a webhook is disabled automatically and stops receiving events entirely. You will see enabled: false and lastStatus: "auto_disabled" on it. Nothing resumes on its own — re-enable it explicitly:

const wh = await client.webhooks.get(webhookId);
if (wh.lastStatus === "auto_disabled") {
  await client.webhooks.update(webhookId, { enabled: true });
}

Re-enabling counts against your plan's webhook limit, so it can fail with QUOTA_EXCEEDED if you are already at the cap.

What's Next?

Examples

Check out the examples directory for runnable code:

  • quick-start.ts - Basic usage
  • worker.ts - Processing jobs
  • workflow-dag.ts - Complex workflows with dependencies
  • grpc-streaming.ts - High-performance gRPC streaming
  • realtime.ts - Real-time event streaming
  • schedules.ts - Cron schedules
  • error-handling.ts - Error handling patterns