title	DamageClaims
emoji	🚀
colorFrom	indigo
colorTo	gray
sdk	docker
app_port	8000
pinned	false
short_description	DamageClaims is a real-world OpenEnv environment

---

DamageClaims

Automated freight insurance adjudication via reinforcement learning.

DamageClaims is a production-ready OpenEnv environment that models the end-to-end claims investigation workflow used by freight insurance adjusters. An LLM agent gathers evidence documents, reasons over multi-party liability, detects fraud, and submits a structured payout decision — all within a strict turn budget.

This is not a toy benchmark. It models operational constraints from real logistics and insurance workflows.

---

The Problem

Freight damage claims are expensive to process manually. A single adjuster must:

• Collect and cross-reference 5–10 documents per claim (bills of lading, inspection reports, delivery receipts, photos)
• Attribute liability across three parties — carrier, warehouse, and shipper — whose responsibilities overlap
• Calculate accurate payouts against declared and verified cargo values
• Flag inflated or fraudulent claims before issuing payment

At scale, this process is slow, inconsistent, and error-prone. DamageClaims trains an agent to handle this automatically — making grounded, auditable decisions from evidence under time pressure.

---

Architecture

flowchart TD
    A[LLM Agent] -->|Builds prompt from observation| B[inference.py]
    B -->|Parses JSON action| C[DamageClaimsEnv Client]
    C -->|HTTP POST /reset, /step| D[FastAPI Server]
    D --> E[DamageClaimsEnvironment]
    E -->|Loads task config| F[data/tasks.json]
    E -->|Scores decision| G[Deterministic Grader]
    G -->|approval + liability + payout + efficiency - doc_penalty| E
    E -->|DamageClaimsObservation| D
    D -->|observation, reward, done| C
    C --> B
    B -->|START, STEP, END logs| H[stdout / CI pipeline]

Loading

Episode Sequence

sequenceDiagram
    participant Agent
    participant Client as EnvClient
    participant Server as FastAPI
    participant Env as DamageClaimsEnvironment

    Agent->>Client: reset(task_id)
    Client->>Server: POST /reset
    Server->>Env: reset()
    Env-->>Client: DamageClaimsObservation (turn=0, done=False)

    loop Max 8 turns
        Agent->>Client: step(action)
        Client->>Server: POST /step
        Server->>Env: step(action)
        Env->>Env: apply action + grade + reward
        Env-->>Client: observation, reward, done
        Client-->>Agent: updated state
    end

    Note over Agent,Env: submit_decision triggers grader and ends episode

Loading

---

Task Suite

Task	Difficulty	Claim ID	Scenario	Ground Truth
simple_carrier_fault	Easy	CLM-001	3 laptops with cracked screens damaged in transit	100% carrier liability
split_liability	Medium	CLM-002	Pharma cold-chain breach during shipping and storage	60% carrier / 40% warehouse
fraud_detection	Hard	CLM-003	Luxury watch shipment with inflated declared value and partial theft	Partial approval, reduced payout

All tasks are deterministic. Each specifies available_documents, critical_documents, ground_truth targets, and question_hints.

---

Scoring

The grader computes a single score in [0.0, 1.0] from five components:

score = approval_score + liability_score + payout_score + efficiency_bonus - doc_penalty

Component	Weight	Condition
Approval correct	+0.40	approved matches ground truth
Liability per party (×3)	+0.10 each	Within 15 percentage points of ground truth
Payout accuracy	+0.20	Relative error ≤ 10% of ground-truth payout
Efficiency bonus	up to +0.10	(MAX_TURNS - turns_used) / (MAX_TURNS - 1)
Document penalty	−0.20 × (missing / total)	Proportional to critical documents not collected

Mid-episode signal (every observation)

{
  "critical_docs_collected": 2,
  "critical_docs_required": 3,
  "turns_remaining": 4
}

Terminal signal (on submit_decision)

{
  "approval": 0.4,
  "liability": 0.3,
  "payout": 0.2,
  "efficiency": 0.071,
  "doc_penalty": -0.0
}

Reward shaping (intermediate steps)

Action	Reward
Valid document requested	+0.05
Document not in available list	−0.05
Question asked	+0.02
submit_decision (terminal)	grade() output
Max turns reached without submit	0.0

---

Action Space

Actions use the DamageClaimsAction model:

Field	Type	Notes
action_type	enum	request_document, ask_question, submit_decision
document_name	string or null	Required for request_document
question	string or null	Required for ask_question
decision	ClaimDecision or null	Required for submit_decision

ClaimDecision fields:

Field	Type	Constraint
approved	bool	Approve or reject claim
payout_usd	float	Dollar amount
carrier_liability_pct	float	Must sum to 100 (±1.0) with others
warehouse_liability_pct	float	—
shipper_liability_pct	float	—

---

Project Structure

.
├── inference.py              # Agent driver: prompt loop, logging, fallback decisions
├── client.py                 # DamageClaimsEnv HTTP client
├── models.py                 # Pydantic action and observation models
├── openenv.yaml              # OpenEnv spec
├── pyproject.toml
├── data/
│   └── tasks.json            # Task definitions, ground truth, document library
├── assets/
│   ├── banner.png
│   └── logo.png
├── scripts/
│   └── validate-submission.sh
└── server/
    ├── app.py                # FastAPI endpoints: /reset /step /state /health
    ├── damageClaims_environment.py  # Core environment + grader
    ├── Dockerfile
    └── requirements.txt

---

Setup and Usage

Prerequisites

• Python 3.10+
• uv or standard pip

Two-column quickstart

Local Dev	Docker
python -m venv .venv	docker build -t damageclaims-env:latest .
source .venv/bin/activate	docker run --rm -p 8000:8000 damageclaims-env:latest
pip install -e .	curl http://127.0.0.1:8000/health
uvicorn server.app:app --host 0.0.0.0 --port 8000	—
uv run python inference.py	—

Environment variables

Copy .env.example and fill in your credentials:

cp .env.example .env

API_BASE_URL=https://api.groq.com/openai/v1
MODEL_NAME=llama-3.3-70b-versatile
HF_TOKEN=your_api_key_here

API_KEY is accepted as a fallback if HF_TOKEN is not set.

---

Testing

Smoke test — single task

curl -sS -X POST http://127.0.0.1:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": "simple_carrier_fault"}'

Full inference run

uv run python inference.py

Expected output format:

[START] task=simple_carrier_fault env=damageClaims model=llama-3.3-70b-versatile
[STEP] step=1 action=request_document reward=0.05 done=false error=null
[STEP] step=2 action=request_document reward=0.05 done=false error=null
[STEP] step=3 action=submit_decision reward=0.53 done=true error=null
[END] success=true steps=3 score=0.525 rewards=0.05,0.05,0.53
--- simple_carrier_fault: score=0.525 ---

=== AVERAGE SCORE: 0.500 ===

OpenEnv validation

.venv/bin/openenv validate

---

Deployment (Hugging Face Spaces)

1. Create a Docker SDK Space.
2. Set Space secrets: API_BASE_URL, MODEL_NAME, HF_TOKEN.
3. Expose port 8000.
4. Run the submission validator after deploy:

bash scripts/validate-submission.sh https://<space_name>-<user_name>.hf.space .

The validator checks three things in order:

1. HF Space live — POST /reset returns HTTP 200
2. Docker build — docker build completes within 600 seconds
3. OpenEnv validate — openenv validate passes in the repo directory

---

Submission Checklist

Item	Status
HF Space responds to /reset	Implemented
openenv.yaml at repo root, spec-compliant	Implemented
Docker builds with health check	Implemented
inference.py reproduces baseline at temperature=0.1	Implemented
3 tasks (easy / medium / hard) with deterministic graders	Implemented
Scores clamped to [0.0, 1.0]	Implemented
Partial progress signal in every observation	Implemented
reset() / step() / state() interface satisfied	Implemented

---

Troubleshooting

Symptom	Fix
openenv validate fails	Confirm openenv.yaml is at repo root and app: server.app:app is importable
Auth error during inference	Verify HF_TOKEN or API_KEY is set in .env
Docker health check fails	docker logs <container_id> — confirm port 8000 is exposed
KeyError on task_id	Valid IDs: simple_carrier_fault, split_liability, fraud_detection
Liability validation error	Ensure the three liability percentages sum to exactly 100
Network issues in container	Check egress proxy settings; x-deny-reason header indicates block reason

---

OpenEnv Interface Compliance

Component	Location
openenv.yaml	Repo root
Action model (DamageClaimsAction)	models.py
Observation model (DamageClaimsObservation)	models.py
API endpoints (/reset, /step, /state, /health)	server/app.py
Environment class	server/damageClaims_environment.py

---

Developers

Built by Prieyan and Vishvak R.