cloud-rag

Provider-abstracted RAG microservice demonstrating AWS Bedrock and GCP Vertex AI integration.

What it is and which gap it closes

This project implements a production-shaped Retrieval-Augmented Generation pipeline that can run against AWS Bedrock (Titan Embed + Claude), GCP Vertex AI (text-embedding-004 + Gemini), a local Ollama instance, or a fully offline mock — all through a single abstract provider interface. The core package is pure Python stdlib with no third-party imports at module level; cloud SDKs are loaded lazily only when real credentials and an explicit opt-in env var are present, so nothing is billed and no install is required to run the demo or tests. This directly closes the "no Bedrock/Vertex hands-on" gap for an AI/ML Engineer role: the adapters are real, idiomatic SDK code — just gated.

Architecture

                        ┌─────────────────────────────┐
                        │        RagPipeline          │
                        │  ingest(docs) → query(q)    │
                        └────────────┬────────────────┘
                                     │  injects
                    ┌────────────────┼────────────────┐
                    │                │                │
           ┌────────▼──────┐  ┌──────▼───────┐       │
           │ EmbeddingProv │  │  LLMProvider │       │
           │   (abstract)  │  │   (abstract) │       │
           └───────┬───────┘  └──────┬───────┘       │
    ┌──────────────┼──────┐   ┌──────┼──────────┐    │
    │              │      │   │      │          │    │
 MockEmbed  DetermEmbed Titan │  MockLLM   Ollama Gemini
  (offline)  (local)  (Bedrock) (offline) (local) (Vertex)
                                    │
                             BedrockClaude

Provider registry (providers/registry.py) selects the pair at startup via $CLOUD_RAG_PROVIDER. Default is mock.

RAG core (rag/) is entirely stdlib: character chunking with overlap, pure-Python cosine similarity, in-memory vector store.

Input guard (rag/pipeline.py) rejects empty and oversized queries at the boundary before any embedding or generation call.

Run for free (mock / local)

# Clone and run — no installs, no credentials, no cost
git clone <repo>
cd cloud-rag
python -m cloud_rag.cli demo

To use a local Ollama model instead:

ollama serve          # in another terminal
ollama pull llama3
CLOUD_RAG_PROVIDER=local python -m cloud_rag.cli demo

Run tests

# pytest (install: pip install pytest)
pytest tests/ -v

# Or run the test files directly if pytest is not available:
python tests/test_pipeline.py    # (use pytest for proper output)

Cloud provider switches

AWS Bedrock

export CLOUD_RAG_PROVIDER=bedrock
export AWS_BEDROCK_ENABLED=true          # explicit opt-in prevents accidental billing
export AWS_DEFAULT_REGION=us-east-1
export AWS_ACCESS_KEY_ID=<your-key>
export AWS_SECRET_ACCESS_KEY=<your-secret>
export BEDROCK_LLM_MODEL_ID=anthropic.claude-3-haiku-20240307-v1:0
export BEDROCK_EMBED_MODEL_ID=amazon.titan-embed-text-v1
pip install boto3
python -m cloud_rag.cli demo

GCP Vertex AI

export CLOUD_RAG_PROVIDER=vertex
export GCP_VERTEX_ENABLED=true           # explicit opt-in
export GOOGLE_CLOUD_PROJECT=my-project
export VERTEX_LOCATION=us-central1
export VERTEX_LLM_MODEL=gemini-1.5-flash-002
export VERTEX_EMBED_MODEL=text-embedding-004
gcloud auth application-default login
pip install google-cloud-aiplatform
python -m cloud_rag.cli demo

Optional REST API

pip install fastapi uvicorn
uvicorn cloud_rag.api.app:app --reload
# POST http://localhost:8000/ingest  {"documents": {"doc.md": "..."}}
# POST http://localhost:8000/query   {"question": "..."}

Responsible-AI notes

• Input validation: queries are validated at the boundary (_validate_query). Empty or oversized (>2000 chars) inputs are rejected with a clear error to mitigate prompt-injection via malformed user input.
• PII / logging: in production, query logs must be stored in an append-only audit system (not general application logs) and scrubbed of personal identifiers (names, account numbers, health data) before retention. See data_security_policy.md in sample_docs for the policy framing.
• Accidental billing guard: cloud providers require AWS_BEDROCK_ENABLED=true or GCP_VERTEX_ENABLED=true at construction time. Without the flag the constructor raises a clear RuntimeError before any network call is made.
• No secrets in code: all credentials are injected via environment variables; no keys are hardcoded.