Summary
Bring statistical anomaly detection functions from CEAP-Playbook into VigiaBR's processing pipeline.
Motivation
VigiaBR has robust data collection infrastructure (7 sources, Scrapy spiders, Airflow DAGs) but zero anomaly detection code. The CEAP-Playbook repository has battle-tested statistical functions that detect expense anomalies:
- Benford's Law — chi-squared test on first-digit distribution
- HHI (Herfindahl-Hirschman Index) — supplier concentration measurement
- Round values detection — flags suspiciously round amounts
These are exactly the building blocks needed to populate VigiaBR's Inconsistencia model and begin SCI scoring.
Current State
pipeline/schemas/models/inconsistencia.py — model exists but nothing generates records
pipeline/processing/processing/transformers/camara.py — skips despesa records (_SKIPPED_TYPES)
- No analysis/detection module exists in the pipeline
Acceptance Criteria
References
Summary
Bring statistical anomaly detection functions from CEAP-Playbook into VigiaBR's processing pipeline.
Motivation
VigiaBR has robust data collection infrastructure (7 sources, Scrapy spiders, Airflow DAGs) but zero anomaly detection code. The CEAP-Playbook repository has battle-tested statistical functions that detect expense anomalies:
These are exactly the building blocks needed to populate VigiaBR's
Inconsistenciamodel and begin SCI scoring.Current State
pipeline/schemas/models/inconsistencia.py— model exists but nothing generates recordspipeline/processing/processing/transformers/camara.py— skipsdespesarecords (_SKIPPED_TYPES)Acceptance Criteria
pipeline/processing/processing/analyzers/_SKIPPED_TYPES)InconsistenciaCreateSchemarecords when anomalies detectedReferences
utils/ceap_utils.py,utils/constantes.py