llmkit (Python)

Unified LLM client library. One API, multiple providers, zero external dependencies.

The code is generated + hand-coded with the help of AI: a typed provider matrix is generated from a single source of truth, while request building, transport, streaming, caching, batching, and tool-loop behavior are hand-coded on top.

Install

pip install llmkit
# or with uv:
uv add llmkit

Python 3.10 or later.

Quick start

import os
import llmkit

resp = llmkit.prompt(
    provider=llmkit.Provider(name="anthropic", api_key=os.environ["ANTHROPIC_API_KEY"]),
    request=llmkit.Request(system="Be concise.", user="Say hi"),
    temperature=0.3,
)
print(resp.text)
print(resp.tokens.input, "input tokens")

Streaming

def on_chunk(text: str) -> None:
    print(text, end="", flush=True)

resp = llmkit.prompt_stream(
    provider=llmkit.Provider(name="openai", api_key=os.environ["OPENAI_API_KEY"]),
    request=llmkit.Request(user="Write a haiku about caching."),
    on_chunk=on_chunk,
)

Tool-calling agent

def weather(args):
    return f"It's sunny in {args['city']}."

agent = llmkit.Agent(
    llmkit.Provider(name="anthropic", api_key=os.environ["ANTHROPIC_API_KEY"]),
)
agent.set_system("You can look up weather with the 'weather' tool.")
agent.add_tool(
    llmkit.Tool(
        name="weather",
        description="Get weather for a city",
        schema={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
        run=weather,
    )
)
resp = agent.chat("What's the weather in Helsinki?")
print(resp.text)

Image generation

Generate images from text, optionally conditioned on reference images for editing or composition. Currently supports Google's Nano Banana 2 (gemini-3.1-flash-image-preview) and Pro (gemini-3-pro-image-preview).

Text-to-image — pass prompt for the terse hot path:

resp = llmkit.generate_image(
    provider=llmkit.Provider(name="google", api_key=os.environ["GOOGLE_API_KEY"]),
    request=llmkit.ImageRequest(
        model="gemini-3.1-flash-image-preview",
        prompt="A nano banana dish in a fancy restaurant",
    ),
    aspect_ratio="16:9",
    image_size="2K",
)
with open("out.png", "wb") as f:
    f.write(resp.images[0].data)

For editing or compositional generation, pass parts — an ordered list of text and image parts. The Text(...) and Image(...) constructors build each part; on-wire ordering matches the list order, so the model attends to descriptions and references in the pairing you intend:

edited = llmkit.generate_image(
    provider=provider,
    request=llmkit.ImageRequest(
        model="gemini-3.1-flash-image-preview",
        parts=[
            llmkit.Text("Person:"),
            llmkit.Image("image/png", person_bytes),
            llmkit.Text("Outfit:"),
            llmkit.Image("image/png", outfit_bytes),
            llmkit.Text("Generate the person wearing the outfit."),
        ],
    ),
)

Set exactly one of prompt or parts — both empty or both set raises ValidationError.

Aspect ratios and sizes are validated against a per-model whitelist before the HTTP request — image_size="512" on Pro raises ValidationError without paying for a 4xx round-trip.

Model	Aspect ratios	Sizes
Nano Banana 2 (Flash)	1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1	512, 1K, 2K, 4K
Nano Banana Pro	1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9	1K, 2K, 4K

Up to 14 reference images per request.

Batching

requests = [llmkit.Request(user=f"Summarize: {text}") for text in corpus]
responses = llmkit.prompt_batch(
    provider=llmkit.Provider(name="anthropic", api_key=key),
    requests=requests,
)

Providers

OpenAI, Anthropic, Google, Grok, Bedrock, OpenRouter, Groq, DeepSeek, Cohere, Mistral, Together, Fireworks, Cerebras, Doubao, Ernie, Moonshot, Qwen, Perplexity, SambaNova, Yi, AI21, Zhipu, MiniMax, Azure, Ollama, LM Studio, vLLM.

Each provider has a default model, auth scheme, and feature matrix (caching, batching, streaming, tool calls, structured output, file upload) discoverable via llmkit.PROVIDERS.

API surface

Entry points: prompt, prompt_stream, upload_file, prompt_batch, submit_batch, wait_batch, Agent.

Types: Provider, Request, Response, Message, File, Image, Tool, Options, Usage, Event, MiddlewareFn, MiddlewareOp, MiddlewarePhase, ProviderName, ProviderConfig, PROVIDERS.

Errors: APIError, ValidationError, MiddlewareVetoError.

License

MIT.