cihjml - Centro de Investigaciones Históricas José María Arboleda Llorente

Data migration and normalization pipeline for the cihjml (Centro de Investigaciones Históricas José María Arboleda Llorente).

Overview

This repository processes and normalizes archival collection data, including:

• Date normalization and validation
• Geographic entity recognition and gazetteer matching
• Collection metadata processing
• Data quality diagnostics

Main Components

• procesar.py - Main processing pipeline
• corrector.py - Data correction modules (dates, places, collections, entities)
• datenormalizer.py - Date parsing and standardization
• diagnostics.py - Data quality checks
• ent_reconocimiento.py - Entity recognition
• gazeetter.py / lugares.py - Geographic place matching

Setup

pip install -r requirements.txt

Data

Input/output CSV files are stored in the data/ directory.