Gaussian Process Prior Variational Autoencoder for Endoscopic Videos

2026-06-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

A new Gaussian Process Prior Variational Autoencoder (GPVAE) framework addresses common degradations in endoscopic videos, such as specular reflections, motion artifacts, and missing frames. This system replaces the typical factorized latent prior with a temporal Gaussian process prior, enabling uncertainty-aware reconstruction and interpolation of missing frames. The GPVAE integrates endoscopy-specific encoders, including a convolutional EndoVAE backbone and pretrained Vision Transformer encoders from GastroNet-5M, alongside Hierarchical Prior Approximation (HPA) and Sparse Precision Approximation (SPA) for scalable Gaussian Process approximations. A DUCKNet-based masking pipeline specifically manages specular reflections. Evaluated on the C3VDv2 colonoscopy dataset, GPVAE variants achieved an average 21.9% reduction in image reconstruction RMSE, up to 26.1%, compared to VAE baselines. Downstream trajectory RMSE saw a 12.7% average reduction, with a 27.3% increase in training time per epoch.

Key takeaway

For Computer Vision Engineers developing robust medical video analysis systems, this GPVAE framework offers a significant advancement. You should consider integrating temporal Gaussian Process priors into your VAE architectures to enhance restoration quality and provide crucial uncertainty estimates. This approach can reduce image reconstruction RMSE by over 20% and improve downstream trajectory accuracy, making your diagnostic and interventional tools more reliable despite increased training time.

Key insights

A Gaussian Process Prior Variational Autoencoder (GPVAE) leverages temporal continuity for robust, uncertainty-aware endoscopic video restoration.

Principles

• Exploit temporal continuity for video sequence restoration.
• Uncertainty estimates provide confidence for restored frames.
• Combine domain-specific encoders with advanced priors.

Method

The GPVAE framework replaces the standard VAE latent prior with a temporal Gaussian process prior, integrating endoscopy-specific encoders and scalable GP approximations, while using DUCKNet for specular reflection masking.

In practice

• Implement temporal GP priors in VAEs for sequential data.
• Integrate per-frame uncertainty for medical image quality.
• Utilize DUCKNet-based masking for specular reflection removal.

Topics

• Gaussian Process
• Variational Autoencoder
• Endoscopic Video Analysis
• Medical Imaging
• Video Restoration
• Uncertainty Estimation

Best for: AI Scientist, Computer Vision Engineer, Research Scientist

Related on AIssential

• GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection — Integrating GP uncertainty with frozen CLIP enhances few-shot OOD detection without fine-tuning.
• Variational autoencoders with latent high-dimensional steady geometric flows for dynamics — VAE-DLM integrates geometric flows into latent spaces for robust learning of PDE dynamics, reducing OOD error.
• SP$^3$: Spherical Priors for Plug-and-Play Restoration — Spherical Encoders can replace denoisers in Plug-and-Play image restoration for faster, high-quality results.
• Closed-form predictive coding via hierarchical Gaussian filters — Closed-form predictive coding via hierarchical Gaussian filters enables deep, efficient, and biologically plausible neural network training.
• The Theory and Practice of Highly Scalable Gaussian Process Regression with Nearest Neighbours — NNGP/GPnn offer scalable, principled Gaussian process regression with minimax-optimal rates and hyperparameter robustness.
• Learning a Sampling-Free Variational DNN Plugin from Tiny Training Sets to Refine OOD Segmentation With Uncertainty Estimation — VarDeepPCA refines OOD medical image segmentation using intrinsic geometric priors and sampling-free variational learning from tiny in-distribution datasets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.