Skip to content

Add multi-provider AI configuration - #148

Open
c-sonnier wants to merge 7 commits into
activerabbit-ai:mainfrom
c-sonnier:main
Open

c-sonnier wants to merge 7 commits into
activerabbit-ai:mainfrom
c-sonnier:main

Conversation

@c-sonnier

Copy link
Copy Markdown
Contributor

Add Multi-Provider AI Configuration

Summary

Adds per-account AI provider configuration, letting each account choose their own AI provider (Anthropic, OpenAI, Google Gemini, etc.) and model for the AI-powered features (error summaries, performance summaries, PR content generation, branch naming, code fix application). Replaces the hardcoded claude-haiku-4-5 / ANTHROPIC_API_KEY-only integration with a pluggable, DB-backed config layered over a unified RubyLLM interface.

Deployments that set ANTHROPIC_API_KEY (or any supported provider's ENV var) continue to work unchanged — accounts without an explicit config fall back to ENV automatically.

Motivation

  • Accounts want to bring their own provider/keys (cost, data residency, model preference).
  • AI services duplicated Anthropic HTTP wiring across four+ classes. A shared RubyLLM-based concern removes that duplication and makes adding providers a config change, not a code change.
  • Single-provider lock-in blocked onboarding customers with existing OpenAI/Gemini contracts.

Screenshots

Screenshot 2026-04-15 at 11 55 34 AM

What's Included

New

  • AiProviderConfig model (account_id, provider, model, encrypted api_key, active) with one active config per account enforced via activate!.
  • Account-scoped CRUD at /account/settings/ai_provider_configs (create, update, destroy, activate, models) with AiProviderConfigsController.
  • Settings UI — Stimulus controller (ai_provider_form_controller.js) with dynamic model list fetched per provider, search/filter, masked key input, loading and failure states.
  • Shared concern AiProviderChat — one place where every AI service acquires a configured RubyLLM chat instance.
  • Migration create_ai_provider_configs — additive only, with FK + index on account_id.
  • Specs for model, request, factory, and updated service specs.

Changed

  • AiSummaryService, AiPerformanceSummaryService, Github::BranchNameGenerator, Github::PrContentGenerator, Github::SimpleCodeFixApplier — all now take account: and delegate provider/key resolution to AiProviderChat.
  • Account#ai_provider_config returns the persisted active config or an ENV-backed transient config if none exists.
  • active_record_encryption.rb initializer now ENV-only, fails fast in production when keys missing (required since AiProviderConfig#api_key is encrypted).
  • SUPER_ADMIN_WRITABLE_CONTROLLERS includes ai_provider_configs so super admins can manage provider configs while viewing another account.

Dependencies

  • ruby_llm 1.14.1 (+ transitive gems in Gemfile.lock).

Backwards Compatibility

  • No breaking changes for existing deployments.
  • ENV-only setup (e.g. ANTHROPIC_API_KEY) continues to work via the Account#ai_provider_config fallback.
  • AiSummaryJob#perform signature unchanged — in-flight jobs from prior code continue to succeed.
  • Migration is additive; no data migration needed.
  • No existing data is encrypted by the new encryption initializer (this is the first model to use encrypts), so ENV vars need to be set only before creating the first AiProviderConfig.

Deployment Notes

Before merging, production needs the three Active Record encryption ENV vars:

ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY
ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY
ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT

Generate with bin/rails db:encryption:init. The initializer raises on boot in production if any are missing.

Testing

  • RSpec: 44/44 target specs pass (model, request, services, account).
  • Minitest: 3/3 remaining AI service tests pass (stale ai_summary_service_test.rb deleted — superseded by RSpec).
  • GitHub service specs: 62/62 pass.
  • Manually exercised the settings UI end-to-end: provider add/update/delete/activate, model dropdown fetch, unconfigured-state messaging, and AI summary generation against two providers.

Test Plan

  • bundle exec rspec passes
  • bundle exec rails test passes
  • Fresh deploy with all three encryption ENV vars set — boots without error
  • Fresh deploy without encryption ENV vars — raises on boot in production (expected)
  • Existing deploy with only ANTHROPIC_API_KEY — AI summary generates without any DB config
  • Create Anthropic config → generate AI summary → verify uses configured model
  • Switch active config to OpenAI → generate AI summary → verify uses OpenAI
  • Super admin viewing another account → can create/activate/delete configs

…ettings UI

Phase 1 of configurable AI provider: users can add, edit, activate,
and delete AI provider configs from Account Settings. API keys are
encrypted via Rails encrypts. First config auto-activates.
…yService

Replace raw Net::HTTP calls with RubyLLM.context for per-request
API key isolation. Service now accepts account: param and reads
the active AI provider config. Uses fast_model for summaries.
…concern

Extract AiProviderChat concern for per-request RubyLLM context setup.
Convert AiSummaryService, BranchNameGenerator, PrContentGenerator,
and SimpleCodeFixApplier from raw Net::HTTP to RubyLLM. All services
now accept account: param and read the active provider config.
Delete all duplicated HTTP client code from AI services.
Account#ai_provider_config falls back to ENV keys (ANTHROPIC_API_KEY,
OPENAI_API_KEY, GEMINI_API_KEY) when no database config exists. Updated
performance view to link to account settings instead of showing raw ENV
key instructions. Updated all GitHub service specs to use account-based
AI provider mocking.
… interface

Replace raw Anthropic HTTP stubs with RubyLLM ai_chat mocks. Pass
account: to constructor matching the refactored service interface.
- Add Account Settings link to admin sidebar navigation
- Fix unclosed div tag that prevented AI provider form submission
- Add searchable model select dropdowns powered by RubyLLM registry
- Add models API endpoint to serve provider-specific model lists
- Fix nil start_line crash in normalize_source_context helper
- Remove duplicate notification preferences card from account settings
- Remove hardcoded encryption key fallbacks; require ENV vars in production and provide clearly-marked dev defaults
- Delete stale AiSummaryService Minitest file (replaced by RSpec suite using the new account-based interface)
- Add ai_provider_configs to SUPER_ADMIN_WRITABLE_CONTROLLERS so super admins can manage provider configs on viewed accounts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant