Forged Stamp Recognizer

EE7204 / EC7205 — Image Processing and Computer Vision
Department of Electrical and Information Engineering, University of Ruhuna

Team Members

Member	ID	Role
S.A.U. Fernando	EG/2021/4511	Data Pipeline & Augmentation Lead
J.S. Karunarathna	EG/2021/4599	Pre-processing & Segmentation Lead
A.D.H. Karunathilake	EG/2021/4604	Geometric Localization & ROI Extraction Lead
M.M.P.L. Manthilaka	EG/2021/4671	Deep Learning & Evaluation Lead

---

System Overview

An automated hybrid computer vision system that verifies organizational document stamps by distinguishing genuine wet-ink impressions from digitally printed forgeries. The system uses a five-stage pipeline:

Raw document scan
    → [Stage 2] HSV segmentation + morphological cleaning
    → [Stage 3] Contour detection + Hough Circle Transform + ROI crop (224×224)
    → [Stage 4] ResNet50 transfer learning classifier
    → Verdict: GENUINE or FORGED

Forensic basis

Genuine wet-ink stamps show random ink diffusion and absorption patterns. Digital forgeries (created via the Scan-Print-Scan method) show periodic halftone dot patterns from laser/inkjet printing. ResNet50's deep convolutional filters learn to discriminate these microscopic texture differences.

---

Project Structure

forged-stamp-recognizer/
├── notebooks/
│   ├── 00_environment_check.ipynb          — Verify dependencies
│   ├── 01_dataset_audit.ipynb              — Dataset statistics and samples
│   ├── 02_preprocessing_segmentation.ipynb — HSV segmentation development
│   ├── 03_geometric_localization.ipynb     — HCT-based ROI detection
│   ├── 03_stamp_segmentation_contours.ipynb— Contour-based ROI detection (improved)
│   ├── 04_generate_roi_dataset.ipynb       — Batch ROI extraction for all images
│   ├── 05_train_classifier.ipynb           — Original training notebook
│   ├── 05_train_classifier_documented.ipynb— Fully documented training notebook ★
│   └── 06_evaluation.ipynb                 — Full evaluation report ★
│
├── src/
│   ├── config.py       — Path configuration
│   ├── classifier.py   — StampClassifier class (full pipeline, importable) ★
│   └── inference.py    — End-to-end inference on raw document images ★
│
├── outputs/            — Generated by running the notebooks
│   ├── roi_dataset_v3/ — Cropped 224×224 stamp ROIs
│   │   ├── genuine/
│   │   └── forged/
│   ├── models/         — Trained model checkpoints
│   ├── figures/        — Evaluation plots
│   └── reports/        — Metrics CSV
│
└── requirements.txt

★ = Added by Member 4

---

Setup

pip install -r requirements.txt

Google Colab workflow (recommended for GPU training)

from google.colab import drive
drive.mount('/content/drive')

# Clone the repo into your Drive or upload manually, then:
%cd /content/drive/MyDrive/forged-stamp-recognizer

---

Running the Pipeline

Step 1 — Data augmentation

Run notebooks/01_dataset_audit.ipynb to verify your dataset structure.

Step 2 — Segmentation

Run notebooks/02_preprocessing_segmentation.ipynb.

Step 3 — ROI extraction

Run notebooks/04_generate_roi_dataset.ipynb. This generates outputs/roi_dataset_v3/.

Step 4 — Train the classifier

Run notebooks/05_train_classifier_documented.ipynb.
This trains the ResNet50 model in two phases and saves the model to outputs/models/.

Step 5 — Evaluate

Run notebooks/06_evaluation.ipynb.
This generates all evaluation figures and saves outputs/reports/metrics_summary.csv.

---

Single-Image Inference

# From command line
python src/inference.py \
    --image path/to/document.png \
    --model outputs/models/stamp_resnet50_final.keras \
    --save  path/to/annotated_output.png

# From Python
from src.classifier import StampClassifier

clf = StampClassifier.load("outputs/models/stamp_resnet50_final.keras")
result = clf.predict_single("path/to/stamp_roi.png")
# {'class': 'genuine', 'confidence': 0.9734, 'raw_prob': 0.0266, 'inference_ms': 14.2}

---

Evaluation Outputs

Figure	Description
confusion_matrix.png	TP / FP / TN / FN breakdown
roc_curve.png	ROC curve with AUC score
pr_curve.png	Precision-Recall curve with Average Precision
confidence_distribution.png	Model confidence by true class
misclassified_gallery.png	Visual analysis of failure cases
inference_speed.png	ms/image distribution on CPU
gradcam_gallery.png	Grad-CAM heatmaps showing model attention
reports/metrics_summary.csv	All scalar metrics in one table

---

Key Technical Decisions

Decision	Choice	Reason
Backbone	ResNet50	Residual connections stabilise fine-tuning on small dataset
Input size	224×224	Matches ResNet50 pre-training resolution
Training phases	2 (frozen → fine-tune)	Prevents catastrophic forgetting
Fine-tune LR	1e-5	10× smaller than head LR; gentle weight adjustment
Layers unfrozen	Top 30	Last two residual blocks; highest semantic specificity
Regularisation	Dropout 0.35 + 0.25	Counters overfitting on ~210 training images
Explainability	Grad-CAM	Confirms model attends to ink texture, not background

---

References

See full reference list in the project proposal (Mini_Project_Proposal_16.pdf).