PAInpainter

Abstract

3D Gaussian inpainting, a critical technique for numerous applications in virtual reality and multimedia, has made significant progress with pretrained diffusion models. However, ensuring multi-view consistency, an essential requirement for high-quality inpainting, remains a key challenge. In this work, we present PAInpainter, a novel approach designed to advance 3D Gaussian inpainting by leveraging perspective-aware content propagation and consistency verification across multi-view inpainted images. Our method iteratively refines inpainting and optimizes the 3D Gaussian representation with multiple views adaptively sampled from a perspective graph. By propagating inpainted images as prior information and verifying consistency across neighboring views, PAInpainter substantially enhances global consistency and texture fidelity in restored 3D scenes. Extensive experiments demonstrate the superiority of PAInpainter over existing methods. Our approach achieves superior 3D inpainting quality, with PSNR scores of 26.03 dB and 29.51 dB on the SPIn-NeRF and NeRFiller datasets, respectively, highlighting its effectiveness and generalization capability.

Method

Figure 2: Framework of PAInpainter.

Key contributions of PAInpainter. We first introduce a novel iterative framework built upon off-the-shelf generative models for 3D gaussian inpainting, integrating proposed adaptive view sampling, multi-view inpainting and post consistency verification. Our closed-loop, self-correcting approach effectively addresses the challenge of multi-view consistency in 3D scene inpainting, substantially improving the fidelity and coherence of the inpainted results.

Figure 3: Pipeline of PAInpainter.

Overview of PAInpainter for multi-view consistent 3D Gaussian inpainting. Our method is built upon the pretrained Stable-Diffusion-2 and incorporates three key components: 1) perspective graph models spatial relationships among cameras to guide adjacent view sampling; 2) inpaint content propagation transmits inpainting content across adjacent views sampled from the perspective graph, providing extra visual priors for diffusion inpainting; 3) consistency verification evaluates inpainted results based on texture and geometric features coherence. The perspective-aware graph sampling contributes to effective content propagation and consistency verification across multiple views.

Results

Novel View Synthesis

Figure 4: The qualitative comparison of novel view synthesis for various advanced 3D Gaussian inpainting methods on the NeRFiller dataset.

Depth rendering

Figure 5: The depth rendering via alpha blending variants. The depth maps demonstrate the geometric coherence across different views of inpainted 3D scenes by our PAInpainter.

Consistency between inpainted anchor and sampled adjacent views

Figure 6: The inpaint content propagation between anchor images and corresponding adjacent images. With our perspective graph sampling strategy, the anchor image provides sufficient and accurate prior to adjacent images to guide consistent multi-view inpainting.

Visualization of dual-feature consistency verification results

Figure 7: Visualization for consistency verification. Red contours delineate mask boundaries and green boxes highlight top-scoring candidates selected for 3DGS optimization. The upper-left number of each candidate represents the consistency score. This module reliably identifies inpainted regions exhibiting both textural and geometric consistency (zoom for details), enhancing performance and robustness.

BibTeX

@misc{cheng2025perspectiveaware3dgaussianinpainting,
      title={Perspective-aware 3D Gaussian Inpainting with Multi-view Consistency}, 
      author={Yuxin Cheng and Binxiao Huang and Taiqiang Wu and Wenyong Zhou and Chenchen Ding and Zhengwu Liu and Graziano Chesi and Ngai Wong},
      year={2025},
      eprint={2510.10993},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2510.10993}
    }