A complete deep learning framework built from scratch using only NumPy — no PyTorch, no TensorFlow, no sklearn for the core learning algorithm.
Every component is implemented manually and explained clearly: neurons, backpropagation, optimizers, regularization, and the full training loop. The goal was to understand what happens inside the black box before using it as a tool.
It is easy to write model.fit(X, y) and get results. It is much harder to explain what actually happens between those two words.
This project was built to answer the questions that libraries abstract away:
- How does a neuron actually compute its output?
- What is backpropagation, mathematically and in code?
- Why does Adam converge faster than SGD?
- What does Dropout actually do during training vs inference?
- How does data leakage silently destroy a model's real-world performance?
Building every piece by hand — and debugging every mistake — answered all of them.
| Component | Details |
|---|---|
| Activations | ReLU, Sigmoid, Tanh, Linear, Softmax — each with forward() and backward() |
| Dense layer | Fully connected layer with He/Xavier initialization, forward and backward pass |
| Loss functions | MSE (regression), BCE (binary classification), CCE + Softmax (multi-class) |
| Model | Sequential container — stacks layers, manages forward and backward pass |
| Optimizers | SGD with momentum, RMSprop, Adam with bias correction |
| Regularization | L1, L2 weight decay, Dropout (training/inference modes) |
| Trainer | Full training loop with batching, validation, early stopping |
- Dataset loaders for Breast Cancer and MNIST
- Preprocessing: normalization, one-hot encoding, image flattening
- Metrics from scratch: accuracy, precision, recall, F1, confusion matrix
- Plots: loss curve, confusion matrix heatmap, ROC curve with AUC
- CLI with four commands:
train,evaluate,predict,visualize - YAML configuration files — change experiments without touching code
- Model checkpointing — save and reload trained models
- Gradio interactive UI — browser-based demo
deeplearnzero/
├── src/
│ ├── nnlibrary/
│ │ ├── activations.py
│ │ ├── layers.py
│ │ ├── losses.py
│ │ ├── model.py
│ │ ├── optimizers.py
│ │ ├── regularizers.py
│ │ ├── trainer.py
│ │ └── neuron.py
│ ├── data/
│ │ ├── loaders.py
│ │ └── preprocessing.py
│ ├── evaluation/
│ │ ├── metrics.py
│ │ └── plots.py
│ └── utils/
│ ├── logger.py
│ ├── config.py
│ └── checkpoint.py
├── src/cli/
│ └── main.py
├── configs/
│ ├── breast_cancer.yaml
│ └── mnist.yaml
├── ui/
│ └── gradio_app.py
├── requirements.txt
└── README.md
git clone https://github.com/ruzaki11/deeplearnzero
cd deeplearnzero
pip install -r requirements.txt# Train on Breast Cancer dataset
python -m src.cli.main train --config configs/breast_cancer.yaml
# Train on MNIST
python -m src.cli.main train --config configs/mnist.yamlpython -m src.cli.main evaluate --model saved_models/breast_cancer_model.pklpython -m src.cli.main visualize --model saved_models/breast_cancer_model.pklpython ui/gradio_app.pyThen open http://127.0.0.1:7860 in your browser.
import numpy as np
from src.nnlibrary.layers import Dense
from src.nnlibrary.model import Model
from src.nnlibrary.trainer import Trainer
# Build a model
model = Model()
model.add(Dense(n_inputs=30, n_neurons=16, activation="relu"))
model.add(Dense(n_inputs=16, n_neurons=8, activation="relu"))
model.add(Dense(n_inputs=8, n_neurons=1, activation="sigmoid"))
model.compile(loss="bce")
# Train
trainer = Trainer(model, optimizer="adam", lr=0.001)
history = trainer.fit(X, y, epochs=100, batch_size=32)- 569 samples, 30 features
- Task: predict malignant vs benign
- Output layer: 1 neuron + sigmoid + BCE loss
- 70,000 samples, 784 features (28×28 flattened)
- Task: classify handwritten digits 0–9
- Output layer: 10 neurons + softmax + CCE loss
Backpropagation is four lines of math
dZ = dA * activation.backward(Z)
dW = X.T @ dZ / batch_size
db = np.sum(dZ, axis=0) / batch_size
dX = dZ @ W.TThat is the entire backward pass of a Dense layer. The chain rule, applied to matrices.
Data leakage is silent and deadly
If the preprocessor is fitted before the train/test split, it has seen the test set. The model appears to perform well. In production it does not. The fix is to always fit preprocessing on training data only — which is why the preprocessor stays unfitted until the Trainer calls it on training folds.
Adam works because it has memory
SGD moves the same size step every time regardless of gradient history. Adam tracks the running mean and variance of each gradient — parameters with consistently large gradients get smaller updates, and parameters with small gradients get relatively larger ones. This is why it converges faster.
The optimal threshold is never 0.5
For F1 score and imbalanced datasets, the default 0.5 probability threshold is almost never optimal. The right threshold depends on the class balance and the specific precision/recall tradeoff you need.
numpy>=1.24
scikit-learn>=1.3
matplotlib>=3.7
pyyaml>=6.0
gradio>=4.0
pandas>=2.0
Scintefic AI Research orginzation - (SAIR) Under the mentorship of Mohammed Seddig.
Awab Hytham Hesham Mustafa github.com/ruzaki11 · awab3990@gmail.com