AI Security & Governance Evaluation Framework (ASGEF)

Research-grade evaluation of ML and LLM security via adversarial attack simulation.

Highlights

• Attack registry — get_attacks("llm" | "ml", config) returns default attack instances (config supplies dataset paths).
• AttackResult (Phase 7) — success_level ∈ {full, partial, none}, success_score ∈ [0, 1]; success is derived (True iff success_score ≥ 0.5). ASR = mean(success_score) (jailbreak is graded; other attacks use 0.0/1.0). Metadata includes prompt, response, sample_id, attack_category where applicable.
• Outcome classification — each attempt is deterministically labeled as:
  • compromised: true security failure (success_level == "full")
  • degraded: robustness/instability failure (success_level in {"strong_partial", "weak_partial"})
  • safe: correct defensive behavior (success_level == "none") This prevents partial/unstable behavior from being misinterpreted as a full compromise.
• Model hierarchy — BaseModel → LLMModel (query) / MLModel (predict, predict_proba)
• Strict execution — model.provider must be exactly openai, local_llm, or local_stub. OpenAI requires an API key; local LLM loads weights at startup (load_on_init) or fails loudly. Attack execute() errors are not swallowed.
• Phase 5 LLM datasets — Structured JSON corpora: asgef/datasets/llm/prompt_injection.json (≥50 samples) and jailbreak.json (≥31 multi-turn sequences). Missing files raise (no inline fallbacks). Regenerate with python asgef/scripts/generate_phase5_llm_datasets.py.
• Dynamic compliance — OWASP LLM Top 10 (GitHub) + NIST AI RMF Core (official NIST.AI.100-1 PDF). Cached under asgef/cache/frameworks/ with 7-day TTL, SHA256 content hashes, and hash verification on read (Phase 5). Legacy cache files without hash must be deleted to upgrade.
• Deterministic compliance mapping — Per–AttackResult keyword ratio matched_eligible / total_eligible; max across results per framework id; threshold default 0.3 (compliance.semantic_matching.min_match_score). Output uses mapped_frameworks: {id, name, score} plus mapping_complete / controls_above_threshold (explicit empty allowed).
• Risk score (Phase 7) — weighted risk score (0–10) over ASR, confidence_drop, severity, and misalignment_risk (from model.instruction_tuned, model.parameter_count_billions, plus deterministic model_id heuristics). Small / non–instruction-tuned / unknown models do not get an artificially low score.
• Response logging — logging.save_responses: true writes asgef/logs/<UTC_timestamp>/prompt_injection_log.json and jailbreak_log.json (array of {sample_id, prompt, response, success_level, score, outcome, attack_category}).
• RAG security tests when enabled
• ML datasets — datasets/ml/sample_ml_data.csv

Safety: Jailbreak JSON includes red-team style scenarios (some adversarially framed). Use only in authorized research / lab environments.

Setup

pip install -r requirements.txt

First evaluation run needs network access to refresh framework caches (unless valid hashed JSON already exists under asgef/cache/frameworks/). NIST parsing requires pypdf.

Configuration

• model.provider: openai | local_llm | local_stub
• model.api_key or OPENAI_API_KEY for OpenAI
• datasets.llm.prompt_injection / datasets.llm.jailbreak — paths relative to the asgef/ directory unless absolute.
• attacks.llm.enabled / attacks.ml.enabled — per model-type lists. Legacy attacks.enabled applies to both if nested keys are omitted.
• model.instruction_tuned / model.parameter_count_billions — optional overrides for misalignment risk (see core/risk_engine/model_capabilities.py).
• risk_scoring.weights.misalignment — weight for the misalignment term (default 2.5).
• logging.save_responses — optional JSON logs under asgef/logs/.

python asgef/scripts/run_attacks.py --system-info   # includes owasp_hash / nist_hash when cached

Multi-LLM benchmark (Phase 6)

Define a shared model: template (HF options, temperature, etc.) plus models: — a non-empty list of entries merged over that template (provider, model_id, per-model overrides).

python asgef/scripts/run_attacks.py --benchmark --model-type llm
python asgef/scripts/run_attacks.py --benchmark --model-type llm --output pretty

• LLM only: --benchmark requires --model-type llm.
• Strict: the first load or evaluation failure aborts the whole benchmark (no skipping).
• Output: aggregated JSON printed to stdout and saved under asgef/results/benchmark_<UTC_timestamp>.json.
• Ranking: ranking[] sorts by lowest risk_score, then lowest global ASR (deterministic tie-break by original order).

Usage

python asgef/scripts/run_attacks.py --model-type llm
python asgef/scripts/run_attacks.py --model-type ml --ml-model pytorch
python asgef/scripts/run_attacks.py --model-type both
python asgef/scripts/run_attacks.py --model-type llm --enable-rag

Output (JSON)

Outcome Classification

ASGEF separates attack outcomes into security compromise vs robustness degradation vs safe behavior:

• compromised: success_level == "full" (the attack achieved a true security failure)
• degraded: success_level in {"strong_partial", "weak_partial"} (model behavior is unstable/partially failing, but not a full compromise)
• safe: success_level == "none" (defenses held)

This is deterministic and avoids treating all “non-safe” behavior as a single axis, improving interpretation of partial failures.

Signal-based evaluation (deterministic)

For text-producing attacks (LLM + RAG), ASGEF additionally extracts auditable boolean signals from the model response and computes:

• compromise_score: weighted indicators of structured/imperative harmful content
• degradation_score: weighted indicators of instability (repetition, prompt echo)
• safety_score: refusal indicator (overrides)

Outcome is then classified deterministically from these signals/scores (no external model calls).

Top-level identity fields:

{
  "provider": "local_llm",
  "model_id": "mistralai/Mistral-7B-Instruct-v0.1",
  "inference_mode": "local_huggingface_inference",
  "metrics": {"ASR": 0.64, "avg_confidence_drop": 0.11},
  "framework_versions": {
    "owasp_last_fetch": "...",
    "nist_last_fetch": "...",
    "owasp_hash": "dbea6973ff96...",
    "nist_hash": "d3721ae2fac5...",
    "entry_count": 14
  },
  "risk": {"risk_score": 5.07, "risk_level": "Medium", "components": {}},
  "compliance": [
    {
      "attack_name": "prompt_injection",
      "risk_level": "High",
      "asr": 0.8,
      "mapped_frameworks": [
        {"id": "LLM01", "name": "Prompt Injection", "score": 0.6667}
      ],
      "mapping_complete": true,
      "controls_above_threshold": true
    }
  ],
  "attacks": [...]
}

Tune compliance.semantic_matching and compliance.risk_thresholds in asgef/config/config.yaml.

Structure

asgef/
├── core/
│   ├── benchmark/
│   │   └── runner.py             # multi-LLM benchmark orchestration
│   ├── compliance/
│   │   ├── framework_loader.py   # fetch, TTL cache, SHA256
│   │   ├── mapper.py             # deterministic keyword scoring
│   │   └── frameworks/
│   ├── datasets/
│   │   └── llm_datasets.py       # strict JSON validation
│   ├── config_validation.py
│   ├── system_info.py
│   └── ...
├── datasets/llm/
│   ├── prompt_injection.json
│   └── jailbreak.json
├── results/                      # benchmark_<timestamp>.json (Phase 6)
├── cache/frameworks/             # owasp_llm_top10.json, nist_ai_rmf.json (+ hash)
├── scripts/
│   ├── run_attacks.py
│   └── generate_phase5_llm_datasets.py

Extending

• Register new attacks in core/attack_engine/registry.py
• Subclass BaseAttack, implement execute → AttackResult (include metadata["response"] when text output exists).
• Add wrappers under model_wrapper/ and wire in scripts/run_attacks.py