The Manifold Alignment Gap

Why Inter-Layer Similarity Does Not Imply Computational Waste in Transformer Inference

Pablo Luciano Rainieri Blasco — Independent Researcher, Tucuman, Argentina

Key Finding

Early exit strategies assume that high cosine similarity between consecutive transformer layers means those layers are redundant. We show this is wrong. Even skipping the last 2 of 24 layers causes perplexity to explode from 28.6 to 480.2 (a 16.8x increase), despite consecutive layers having cosine similarity > 0.95.

We call this the Manifold Alignment Gap: small per-layer rotations are locally invisible (high pairwise similarity) but compound into large cumulative displacement from the output manifold that the projection head expects.

Results at a Glance

Exit Layer	Perplexity	vs Full Model
24 (full)	28.6	baseline
22	480.2	+1,581%
20	3,958	+13,754%
18	28,964	+101,268%

Cross-Architecture Comparison (threshold 0.95)

Model	Architecture	Structural Redundancy
Qwen2.5-0.5B	Transformer	37.6%
Qwen2.5-7B	Transformer	28.2%
Qwen3.5-0.8B	Hybrid (DeltaNet)	10.3%

Implications for ASIC Design

• Simple layer bypass is insufficient for quality-preserving early exit
• Hardware needs multi-exit projection heads or speculative execution
• Hybrid architectures are closing the redundancy window (37.6% -> 10.3%)
• The optimization target is shrinking as architectures evolve

Repository Structure

paper/
  Paper_01_final.tex       # Full paper (IEEE format, LaTeX)
  Paper_01_final.pdf       # Compiled PDF
  figures/                 # All 7 figures (PDF + PNG)

src/
  profile_tokens.py        # Token profiling across model layers
  train_predictor.py       # Random Forest depth predictor
  early_exit_quality.py    # Perplexity vs exit layer measurement
  asic_sim.c              # Cycle-accurate pipeline simulator
  fix_csv.py              # Saturation criterion fix (3 consecutive layers)
  memory_model.py         # Memory bandwidth analysis
  generate_figures.py     # Reproduce all paper figures

data/
  early_exit_quality.csv   # Perplexity measurements
  simulation_results.csv   # Pipeline simulation output

Reproducing Results

Requirements

• Python 3.10+
• PyTorch with CUDA
• transformers, datasets, pandas, numpy, scikit-learn, nltk, matplotlib
• NVIDIA GPU with >= 8GB VRAM (tested on RTX 3070)
• GCC for the C simulator

Steps

# 1. Profile tokens (generates token_profiles.csv)
python src/profile_tokens.py --num_passages 100

# 2. Fix saturation criterion
python src/fix_csv.py

# 3. Train predictor
python src/train_predictor.py --csv token_profiles_fixed.csv --threshold 0.99

# 4. Measure early exit quality (THE key experiment)
python src/early_exit_quality.py

# 5. Run pipeline simulation
gcc -o asic_sim src/asic_sim.c && ./asic_sim token_profiles_fixed.csv

# 6. Generate figures
python src/generate_figures.py

Citation

If you use this work, please cite:

@article{rainieri2026manifold,
  title={The Manifold Alignment Gap: Why Inter-Layer Similarity Does Not Imply Computational Waste in Transformer Inference},
  author={Rainieri Blasco, Pablo Luciano},
  year={2026},
  note={Preprint}
}

Acknowledgments

This research was conducted with the assistance of Claude Opus 4.6 (Anthropic), used for data analysis, experimental design consultation, and manuscript preparation.

License

CC BY 4.0 — Creative Commons Attribution 4.0 International