A curated collection of papers, benchmarks, and resources on image-guided spatial transcriptomics and vision-based omics analysis.
Last updated: 07/17/2026
This repo aims to record a comprehensive review of existing vision-driven spatial transcriptomics (ST) models, covering methodological background, core architectures, and downstream applications, highlighting the evolution of the field and the connections across different research directions.
We strongly encourage researchers who would like to promote their fantastic work to the vision-omics survey community to make a pull request to update their paper’s information!
Our paper has been accepted by Briefings in Bioinformatics!
@article{zhu2026comprehensive,
title={A comprehensive survey of computer vision methods for spatial transcriptomics},
author={Zhu, Junchao and Deng, Ruining and Guo, Junlin and Yao, Tianyuan and Lu, Siqi and Qu, Chongyu and Xiong, Juming and Zhu, Yanfan and Lu, Zhengyi and Yang, Yuechen and others},
journal={Briefings in Bioinformatics},
volume={27},
number={3},
pages={bbag255},
year={2026},
publisher={Oxford University Press}
}
📘Summary of vision-driven models and their downstream tasks can be found in Table. 1 in our survey paper.
[Biorxiv 2020/Nature Communication 2023] Robust mapping of spatiotemporal trajectories and cell–cell interactions in healthy and diseased tissues [paper] [code]
[Nature Method 2021] SpaGCN: Integrating gene expression, spatial location and histology to identify spatial domains and spatially variable genes by graph convolutional network [paper] [code]
[Biorxiv 2022] conST: an interpretable multi-modal contrastive learning framework for spatial transcriptomics [paper] [code]
[Briefings in Bioinformatics 2024] Innovative super-resolution in spatial transcriptomics: a transformer model exploiting histology images and spatial gene expression [paper] [code]
[Biorxiv 2024] Spatial Transcriptomics Prediction from Histology Images at Single-cell Resolution using RedeHist [paper] [code]
[Genome biology 2024] iIMPACT: integrating image and molecular profiles for spatial transcriptomics analysis [paper] [code]
[Briefings in Bioinformatics 2024] A multi-view graph contrastive learning framework for deciphering spatially resolved transcriptomics data [paper] [code]
[Biorxiv 2024] stFormer: a foundation model for spatial transcriptomics [paper] [code]
[Computational and Structural Biotechnology Journal 2024] Deep clustering representation of spatially resolved transcriptomics data using multi-view variational graph auto-encoders with consensus clustering [paper] [code]
[Research 2025] Deep Learning-Enabled Integration of Histology and Transcriptomics for Tissue Spatial Profile Analysis [paper] [code]
[Nature Communication 2025] STAIG: Spatial transcriptomics analysis via image-aided graph contrastive learning for domain exploration and alignment-free integration [paper] [code]
[Nature Method 2025] A visual-omics foundation model to bridge histopathology with spatial transcriptomics [paper] [code]
[Genome biology 2025] spaMGCN: a graph convolutional network with autoencoder for spatial domain identification using multi-scale adaptation [paper] [code]
[Briefings in Bioinformatics 2025] SpaICL: image-guided curriculum strategy-based graph contrastive learning for spatial transcriptomics clustering [paper] [code]
[Nature Method 2025] Spatial gene expression at single-cell resolution from histology using deep learning with GHIST [paper] [code]
[Bioinformatics 2026] Representation learning for multi-modal spatially resolved transcriptomics data [paper] [code]
[Biorxiv 2021] Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors [paper] [code]
[Scientific Reports 2022] Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation [paper] [code]
[Briefings in Bioinformatics 2024] Innovative super-resolution in spatial transcriptomics: a transformer model exploiting histology images and spatial gene expression [paper] [code]
[BIBM 2024] High-Resolution Spatial Transcriptomics from Histology Images using HisToSGE [paper] [code]
[Nature Communication 2025] STAIG: Spatial transcriptomics analysis via image-aided graph contrastive learning for domain exploration and alignment-free integration [paper] [code]
[CVPR 2025] ASIGN: Anatomy-aware Spatial Imputation Graphic Network for 3D Spatial Transcriptomics [paper] [code]
[Nature Method 2025] A visual-omics foundation model to bridge histopathology with spatial transcriptomics [paper] [code]
[Scientific Reports 2025] Image guided construction of a common coordinate framework for spatial transcriptome data
[Genome biology 2025] spaMGCN: a graph convolutional network with autoencoder for spatial domain identification using multi-scale adaptation [paper] [code]
Current learning paradigms mainly include three forms: (A) Regression-based prediction which directly maps histology tiles to gene expression profiles; (B) Retrieval-based searching that aligns image–gene representations and retrieves nearest reference spots to form predictions; (C) Diffusion-based generation which performs conditional noise prediction with a DiT and, at inference, denoises from noise to generate expression maps.
[Nature Biomedical Engineering 2020] Integrating spatial gene expression and breast tumour morphology via deep learning [paper] [code]
[Biorxiv 2021] Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors [paper] [code]
[Scientific Reports 2022] Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation [paper] [code]
[Briefings in Bioinformatics 2022] Spatial Transcriptomics Prediction from Histology jointly through Transformer and Graph Neural Networks [paper] [code]
[WACV 2023] Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction [paper] [code]
[CVPR 2023] SEPAL: Spatial Gene Expression Prediction from Local Graphs [paper] [code]
[Scientific Reports 2023] Breast cancer histopathology image-based gene expression prediction using spatial transcriptomics data and deep learning [paper] [code]
[Pattern Recognition 2024] Spatial Transcriptomics Analysis of Gene Expression Prediction using Exemplar Guided Graph Neural Network [paper] [code]
[Briefings in Bioinformatics 2024] THItoGene: a deep learning method for predicting spatial transcriptomics from histological images [paper] [code]
[Methods 2024] HistoSPACE: Histology-inspired spatial transcriptome prediction and characterization engine [paper] [code]
[arXiv 2024] M2ORT: many-to-one regression transformer for spatial transcriptomics prediction from histopathology images [paper] [code]
[Medical Image Analysis 2024] Transformer with Convolution and Graph-Node co-embedding: An accurate and interpretable vision backbone for predicting gene expressions from local histopathological image [paper] [code]
[CVPR 2024] Accurate Spatial Gene Expression Prediction by integrating Multi-resolution features [paper] [code]
[Bioinformatics 2024] HE2Gene: image-to-RNA translation via multi-task learning for spatial transcriptomics data [paper] [code]
[Biorxiv 2024] Spatial Transcriptomics Prediction from Histology Images at Single-cell Resolution using RedeHist [paper] [code]
[arXiv 2024] Spatially resolved gene expression prediction from histology via multi-view graph contrastive learning with HSIC-bottleneck regularization [paper]
[BIBM 2024] High-Resolution Spatial Transcriptomics from Histology Images using HisToSGE [paper] [code]
[Research Square 2024] ResSAT: Enhancing Spatial Transcriptomics Prediction from H&E- Stained Histology Images with Interactive Spot Transformer [paper]
[Briefings in Bioinformatics 2024] Gene expression prediction from histology images via hypergraph neural networks [paper] [code]
[Biorxiv 2024] PathOmCLIP: Connecting tumor histology with spatial gene expression via locally enhanced contrastive learning of Pathology and Single-cell foundation model [paper]
[arXiv 2025] Spatial Transcriptomics Analysis of Spatially Dense Gene Expression Prediction[paper]
[CVPR 2025] ASIGN: Anatomy-aware Spatial Imputation Graphic Network for 3D Spatial Transcriptomics [paper] [code]
[Nature Method 2025] Spatial gene expression at single-cell resolution from histology using deep learning with GHIST [paper] [code]
[MIDL 2025] MagNet: Multi-Level Attention Graph Network for Predicting High-Resolution Spatial Transcriptomics [paper] [code]
[Briefings in Bioinformatics 2025] Spatial histology and gene-expression representation and generative learning via online self-distillation contrastive learning [paper] [code]
[Briefings in Bioinformatics 2025] stImage: a versatile framework for optimizing spatial transcriptomic analysis through customizable deep histology and location informed integration [paper] [code]
[Nature Method 2025] Spatial gene expression at single-cell resolution from histology using deep learning with GHIST [paper] [code]
[medRxiv 2025] DeepSpot: Leveraging Spatial Context for Enhanced Spatial Transcriptomics Prediction from H&E Images [paper] [code] [NeurIPS 2025] DeepSpot2Cell: Predicting Virtual Single-Cell Spatial Transcriptomics from H&E images using Spot-Level Supervision [paper] [code] [medRxiv 2026] DeepSpot-M: a multimodal foundation model for transcriptome-wide virtual spatial transcriptomics from histology [paper] [code]
[NeurIPS 2023] Spatially resolved gene expression prediction from histology images via bi-modal contrastive learning [paper] [code]
[ISBRA 2024] Spatial Gene Expression Prediction from Histology Images with STco [paper] [code]
[Briefings in Bioinformatics 2024] Multimodal contrastive learning for spatial gene expression prediction using histology images [paper] [code]
[Nature Method 2025] A visual-omics foundation model to bridge histopathology with spatial transcriptomics [paper] [code]
[Briefings in Bioinformatics 2025] DANet: spatial gene expression prediction from H&E histology images through dynamic alignment [paper] [code]
[Briefings in Bioinformatics 2024] stDiff: a diffusion model for imputing spatial transcriptomics through single-cell transcriptomics [paper] [code]
[ICLR 2025] Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images [paper] [code]
[arXiv 2025] Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching [paper] [code]
[arXiv 2025] HaDM-ST: Histology-Assisted Differential Modeling for Spatial Transcriptomics Generation [paper]
Summary of representative spatial transcriptomics datasets used in image-to-ST studies.
For each dataset, we report the release date, dataset name, profiling resolution, primary organs, number of samples and slides, total spots, and typical applications.
| Year-Mon | Dataset | Title | Resolution | Organ | Sample | Slides | Spot | Application |
|---|---|---|---|---|---|---|---|---|
| 2016-Jul | MOB | Visualization and analysis of gene expression in tissue sections by spatial transcriptomics [paper] | Spot-level | Brain | 1 | 12 | 4,147 | ST Prediction |
| 2020-Jul | cSCC [Source] | Multimodal analysis of composition and spatial architecture in human squamous cell carcinoma [paper] | Spot-level | Skin | 4 | 12 | 8,671 | Alignment/3D, ST Prediction |
| 2020-Aug | Breast-ST [Source] | Integrating spatial gene expression and breast tumour morphology via deep learning [paper] | Spot-level | Breast | 23 | 68 | 30,655 | Alignment/3D, ST Prediction |
| 2021-Feb | DLPFC [Source] | Transcriptome-scale spatial gene expression in the human dorsolateral prefrontal cortex [paper] | Spot-level | Brain | 3 | 12 | 47,681 | Clustering, Alignment/3D, ST Prediction |
| 2021-Oct | HER2 [Source] | Spatial deconvolution of HER2-positive breast cancer delineates tumor-associated cell type interactions [paper] | Spot-level | Breast | 8 | 36 | 13,620 | Alignment/3D, ST Prediction |
| 2023-Jul | Human-Kidney [Source] | An atlas of healthy and injured cell states and niches in the human kidney[paper] | Spot-level | Kidney | 24 | 24 | 74,220 | ST Prediction |
| 2023-Jul | Human-CCS [Source] | Spatially resolved multiomics of human cardiac niches [paper] | Spot-level | Heart | 25 | 41 | 100,539 | ST Prediction, Clustering |
| 2023-Nov | Breast-HD [Source] | Visium HD enables spatial discovery in FFPE human breast cancer at single-cell scale [paper] | Bin-level | Breast | 4 | 4 | - | ST Prediction |
| 2023-Dec | BC1, BC2 [Source] | High resolution mapping of the tumor microenvironment using integrated single-cell, spatial and in situ analysis [paper] | Spot-level | Breast | 2 | 2 | 6,331 | ST Prediction |
| 2023-Dec | Breast-Xenium [Source] | High resolution mapping of the tumor microenvironment using integrated single-cell, spatial and in situ analysis [paper] | Cell-level | Breast | 3 | 3 | - | ST Prediction |
| 2023-Dec | CRC-ST [Source] | Molecular cartography uncovers evolutionary and microenvironmental dynamics in sporadic colorectal tumors [paper] | Spot-level | Bowel | 31 | 41 | 117,216 | ST Prediction, Clustering |
| 2024-Apr | SpatialTME [Source] | The Web-Based Portal SpatialTME Integrates Histological Images with Single-Cell and Spatial Transcriptomics to Explore the Tumor Microenvironment [paper] | Spot-level | Multi-organ (12) | 26 cohort | 296 | - | Multi-tasks |
| 2024-Jun | HEST-1k [Source] | HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis [paper] | Multi-level | Multi-organ (26) | 153 cohort | 1,229 | 2.1M | Multi-tasks |
| 2024-Jun | STImage-1K4M [Source] | STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics [paper] | Multi-level | Multi-tissue (50) | 121 cohort | 1,149 | 4.2M | Multi-tasks |
| 2025-Mar | Mouse-Brain-Xenium [Source] | Optimizing Xenium In Situ data utility by quality assessment and best-practice analysis workflows [paper] | Cell-level | Brain | 1 | 4 | - | ST Prediction |
| 2025-May | OmiCLIP-ST [Source] | A visual–omics foundation model to bridge histopathology with spatial transcriptomics [paper] | Multi-level | Multi-organ (32) | 113 cohort | - | 2.2M | Multi-tasks |
| 2025-Jun | CRC-HD [Source] | High-definition spatial transcriptomic profiling of immune cell populations in colorectal cancer[paper] | Bin-level | Bowel | 5 | 5 | - | ST Prediction |
| 2026-Jun | TCGA-vST [Source] | DeepSpot-M: a multimodal foundation model for transcriptome-wide virtual spatial transcriptomics from histology [paper] | Spot-level (predicted) | Multi-organ (32) | - | ~28.7k | ~296M | Virtual ST |
| 2026-Jun | HEST-Xenium-vST [Source] | DeepSpot-M: a multimodal foundation model for transcriptome-wide virtual spatial transcriptomics from histology [paper] | Cell-level (predicted) | Multi-organ (14) | - | 59 | ~13.3M | Virtual ST |
Common evaluation metrics used in image-to-ST.
Each metric is shown with its full name, category, and mathematical definition.
| Metric | Full Name | Category | Formula |
|---|---|---|---|
| MSE | Mean Squared Error | Error | |
| MAE | Mean Absolute Error | Error | |
| PCC | Pearson Correlation Coefficient | Correlation | |
| SSIM | Structural Similarity Index Measure | Correlation | |
| RVD | Relative Variance Distance | Variation | |
| ARI | Adjusted Rand Index | Clustering | |
| NMI | Normalized Mutual Information | Clustering |
Notes:
-
$y_i, \hat{y}_i$ denote ground-truth and prediction;$\bar{y}, \bar{\hat{y}}$ are their means. -
$\mu_x, \mu_y, \sigma_x^2, \sigma_y^2, \sigma_{xy}$ are local statistics;$c_1, c_2$ are constants. -
$\sigma_{\text{pred}}^{2,j}, \sigma_{\text{gt}}^{2,j}$ are variances of gene$j$ ;$C$ is the number of genes. -
$n_{ij}$ is the contingency table entry, with row sums$n_{i\cdot}$ , column sums$n_{\cdot j}$ , and$n$ the total number of samples. -
$I(U;V)$ is mutual information, and$H(\cdot)$ denotes entropy. - ARI can equivalently be computed as: \
$\mathrm{ARI} = \tfrac{\sum_{ij}\binom{n_{ij}}{2} - \big[\sum_i \binom{n_{i\cdot}}{2}\sum_j \binom{n_{\cdot j}}{2}\big]/\binom{n}{2}}{\tfrac{1}{2}\big[\sum_i \binom{n_{i\cdot}}{2} + \sum_j \binom{n_{\cdot j}}{2}\big] - \big[\sum_i \binom{n_{i\cdot}}{2}\sum_j \binom{n_{\cdot j}}{2}\big]/\binom{n}{2}}$ .