ExtraGS: Geometric-Aware Trajectory Extrapolation with Uncertainty-Guided Generative Priors

Kaiyuan Tan1,2, Yingying Shen2, Haohui Zhu2, Zhiwei Zhan2, Shan Zhao2, Mingfei Tu2,
Hongcheng Luo2, Haiyang Sun2,†, Bing Wang2,✉, Guang Chen2, Hangjun Ye2
1UIUC    2Xiaomi EV   

†Project Leader. ✉Corresponding Author.
Paper Code Cite

Abstract

Synthesizing extrapolated views from recorded driving logs is critical for simulating driving scenes for autonomous driving vehicles, yet it remains a challenging task. Recent methods leverage generative priors as pseudo ground truth, but often lead to poor geometric consistency and over-smoothed renderings. To address these limitations, we propose ExtraGS, a holistic framework for trajectory extrapolation that integrates both geometric and generative priors. At the core of ExtraGS is a novel Road Surface Gaussian(RSG) representation based on a hybrid Gaussian–Signed Distance Function (SDF) design, and Far Field Gaussians (FFG) that use learnable scaling factors to efficiently handle distant objects. Furthermore, we develop a self-supervised uncertainty estimation framework based on spherical harmonics that enables selective integration of generative priors only where extrapolation artifacts occur. Extensive experiments on multiple datasets, diverse multi-camera setups, and various generative priors demonstrate that ExtraGS significantly enhances the realism and geometric consistency of extrapolated views, while preserving high fidelity along the original trajectory.



Pipeline

Left: Our method decomposes driving scenes into specialized 3D Gaussian nodes based on their distinct geometric properties. A novel Road Surface Gaussian is proposed to capture the road surface geometry with a dimension-reduced SDF, and the appearance with planar 3D Gaussians. Right Top: We propose an uncertainty estimation method that considers both occlusion and view deviation. Right Bottom: We leverage off-the-shelf generative models to generate pseudo ground truth at extrapolated views. The estimated uncertainty provides pixel level control to the information from the generative model.



Visualization of Estimated Uncertainty

Left: Noisy rendering at an extrapolated trajectory. Right: Estimated uncertainty. ExtraGS captures two types of uncertainty in extrapolated views: Occlusion and Deviation. This uncertainty guides the generative model to selectively synthesize content only at highly uncertain pixels.



Visual Comparisons

6-camera Extrapolation Results on the NuScenes Dataset. All trajectories are laterally shifted by 3m. We take OmniRe as our baseline method.

1-cam Extrapolation Results on the Waymo Open Dataset. We compare with state-of-the-art method for Reconstruction-Only(OmniRe) and Reonstruction+Generation(Street Crafter). All trajectories are laterally shifted by 3m.

1-cam Extrapolation Results on the NuScenes Dataset. We compare our results with Recondreamer and Recondreamer++. We follow Recondreamer++ and present result on scene-0105, timestep 130-169 and laterally shift the trajectory by 3m.

BibTeX


      @misc{extrags2025,
        title={ExtraGS: Geometric-Aware Trajectory Extrapolation with Uncertainty-Guided Generative Priors}, 
        author={Kaiyuan Tan and Yingying Shen and Haohui Zhu and Zhiwei Zhan and Shan Zhao and Mingfei Tu and Hongcheng Luo and Haiyang Sun and Bing Wang and Guang Chen and Hangjun Ye},
        year={2025},
        eprint={2508.15529},
        archivePrefix={arXiv},
        primaryClass={cs.CV},
        url={https://arxiv.org/abs/2508.15529}, 
      }