# resolVI: A Comprehensive Guide to Correcting Spatial Transcriptomics Segmentation Errors Using Deep Generative Modeling
Spatial transcriptomics has revolutionized our understanding of tissue organization by allowing researchers to measure gene expression while retaining spatial context. However, one of the most persistent challenges in this field is spatial segmentation error — the misassignment of transcripts to cells due to technical artifacts, overlapping cellular boundaries, and diffusion of mRNA molecules. A new computational framework called resolVI has been developed to address this problem by modeling and correcting these errors using a deep generative approach.
## What is resolVI?
resolVI is a Python-based probabilistic model designed to estimate and correct segmentation-induced errors in spatial transcriptomics data. It is implemented within the widely used scvi-tools ecosystem and integrates seamlessly into both Python and R workflows. Built on top of Pyro — a probabilistic programming library that enables GPU-accelerated computation — resolVI uses a sophisticated generative model to decompose observed gene expression into three key components: true biological expression, diffusion artifacts from neighboring cells, and background noise.
## The Generative Model Behind resolVI
At the core of resolVI is a hierarchical generative model that describes the observed expression of each gene in each cell as arising from multiple sources. The model assumes that the observed expression (x_{ng}) for cell (n) and gene (g) is drawn from an underlying state defined by:
– **True expression** ((z_n)): The latent embedding representing the cell’s genuine biological state.
– **Diffusion component**: Gene expression contributions from neighboring cells (N(n)) that were incorrectly segmented.
– **Background expression** ((bg_g)): Unspecific noise from the tissue microenvironment.
The model uses a mixture of Gaussians prior over latent embeddings, allowing it to discover distinct cell states in an unsupervised manner. Each cell is assigned to one of (K) latent clusters through a categorical distribution, and the parameters of the Gaussian mixtures — means, variances, and cluster proportions — are learned using maximum likelihood estimation.
A key innovation is the use of a diffusion encoder that takes the raw gene expression of each cell and learns mixture proportions ((alpha_{n0}) through (alpha_{n2})) that represent the relative contributions of background, diffusion, and true expression. These proportions sum to one, ensuring a principled decomposition of the observed signal.
## How Spatial Context is Incorporated
resolVI leverages the spatial coordinates of cells to model the likelihood of diffusion between neighboring cells. A radial basis function kernel is used to define a Dirichlet prior over diffusion contributions, where cells that are physically closer have a higher probability of contributing to segmentation errors. By default, the model considers the 20 nearest neighbors for each cell, with a learnable scale parameter controlling the spatial decay of these contributions.
## Data Augmentation for Low-Count Cells
Spatial transcriptomics data often contains a large number of cells with very low molecular counts. Standard deep learning encoders struggle to extract meaningful information from these sparse measurements. resolVI addresses this by implementing a data augmentation strategy during training: it resamples expression counts using multinomial sampling from a log-normal distribution parameterized by the median and standard deviation of library sizes across all cells. This encourages the model to learn robust representations even for cells with minimal detected transcripts.
## Model Variants and Output Options
resolVI provides five distinct model variants for different analytical needs:
1. **Unconditioned model**: Operates without observed gene expression, useful for prior analysis.
2. **Generic model**: The standard model that incorporates observed gene expression.
3. **Corrected model**: Sets background and diffusion proportions to zero and true expression to one, generating unbiased gene expression estimates.
4. **Residual model**: Sets true expression to zero, yielding estimates of the diffusion and background components.
5. **Simplified model**: Blocks most variables except estimated counts and proportions, optimized for faster differential expression computation.
Two types of corrected counts are provided: generated counts from the corrected model and composite corrected counts computed by combining generated counts with residual counts and observed counts in a ratio-based formula that does not impute expression.
## Supervised and Transfer Learning Capabilities
resolVI supports supervised scenarios where researchers can provide prior knowledge about cell-type assignments. In this mode, the mixture of Gaussians prior is aligned with known cell-type annotations, encouraging the model to organize latent space according to biological categories. Additionally, a shallow linear classifier trained on the latent embeddings can simultaneously predict cell-type labels during model training, yielding improved separation of cell types compared to approaches used in related methods.
Transfer learning functionality, adapted from the scArches framework, allows resolVI models trained on reference datasets to be applied to new query datasets. This is particularly valuable when working with sequential experiments or when integrating data across different technologies or time points.
## Differential Expression and Niche Analysis
resolVI includes specialized tools for downstream analyses. For differential expression, the model uses importance sampling over the latent space to compute empirical marginal distributions over predicted true expression, with a weighting scheme that favors well-segmented cells. A pseudocount is added to moderate log fold changes for lowly expressed genes while preserving unbiased estimates of the fold change itself.
For differential niche abundance analysis, the model randomly samples from the latent codes of neighboring cells, predicts cell-type probabilities using the trained classifier, and performs weighted spatial averaging to produce cell-type frequency vectors that capture the local tissue microenvironment.
## Size Normalization and Batch Effects
An alternative modeling scheme within resolVI incorporates physical cell size measurements (area or volume) as a per-cell scaling factor. This accounts for variation in RNA molecule density that may arise from biological differences in cell states rather than technical detection differences. Background parameters are also learned separately for each batch, enabling the model to handle systematic technical variation across experiments.
## Implementation Details
resolVI is implemented within the scvi-tools library using Pyro as its backend. The default architecture uses ten latent dimensions, encoder networks with two hidden layers of 128 nodes each, and decoder networks with two hidden layers of 32 nodes each. A nearest-neighbor graph is constructed using scanpy and scikit-learn to assign spatial neighbors. The model employs a custom warmup schedule over just 20 epochs, with an increased learning rate applied to nonamortized parameters to improve convergence and correction quality.
## Benchmarking and Evaluation
resolVI has been evaluated across multiple spatial transcriptomics technologies, including 10x Xenium, Vizgen MERSCOPE, and Nanostring CosMx. Integration metrics from scib-metrics are used to benchmark performance, with PCR comparison disabled to avoid misleading results after artifact correction. Double-positive metrics derived from single-cell reference datasets provide an additional layer of validation by identifying genes that appear co-expressed in spatial data but not in single-cell references — a strong indicator of segmentation artifacts.
## FAQ
**What types of spatial transcriptomics data can resolVI analyze?**
resolVI has been validated on data generated by 10x Xenium, Vizgen MERSCOPE, and Nanostring CosMx technologies. Its probabilistic framework is technology-agnostic and can be applied to any spatial transcriptomics dataset where cell segmentation and gene expression measurements are available.
**Does resolVI require cell-type annotations to work?**
No, resolVI can operate in an unsupervised manner using a mixture of Gaussians prior to discover cell states without any prior annotations. However, it also supports supervised scenarios where cell-type labels can be provided to improve latent space organization and enable simultaneous cell-type classification.
**How does resolVI handle cells with very low gene counts?**
resolVI implements a data augmentation strategy that resamples low-count expression data during training. It also uses a low filtering threshold of 20 molecules per cell to retain biologically relevant cells that might otherwise be discarded. These measures ensure that cells with sparse measurements are still properly modeled and corrected.
**Can resolVI be used for longitudinal or multi-condition experiments?**
Yes, resolVI supports transfer learning, allowing models trained on one dataset to be applied to new query datasets. This makes it suitable for longitudinal studies, batch corrections, and cross-condition comparisons.
**What is the difference between generated corrected counts and composite corrected counts?**
Generated corrected counts come directly from the model variant that sets background and diffusion to zero. Composite corrected counts are computed using a ratio formula that combines generated counts, residual counts, and observed counts. The composite approach does not impute expression, while the generated counts do provide imputed values for genes with low detection.
**Is resolVI compatible with other analysis tools in the single-cell ecosystem?**
Yes, resolVI is built within scvi-tools and integrates with tools such as scanpy, squidpy, and scArches. Its outputs can be directly used with differential expression frameworks, clustering algorithms, and visualization tools commonly employed in spatial transcriptomics analysis.
**How does resolVI compare to other segmentation correction methods?**
resolVI differs from segmentation-only correction tools like xeniumresegment, Baysor, and ProSeg by providing a probabilistic decomposition of expression into biological signal, diffusion artifacts, and background noise. Rather than simply reassigning transcripts to cells, it models the generative process that gives rise to observed expression patterns, allowing for more nuanced correction and quantification of uncertainty.
## Conclusion
resolVI represents a significant advancement in the correction of spatial transcriptomics segmentation errors. By modeling the underlying generative process through a principled probabilistic framework, it provides researchers with both corrected expression values and interpretable decompositions of technical artifacts. Its integration into the scvi-tools ecosystem ensures broad accessibility and compatibility with established single-cell and spatial analysis workflows. The model’s support for supervised learning, transfer learning, differential expression, and niche abundance analysis makes it a versatile tool for the spatial transcriptomics community. As spatial technologies continue to evolve and produce increasingly complex datasets, generative approaches like resolVI will be essential for extracting accurate biological insights from spatially resolved measurements.
Thank you for reading
