文献类型： 会议论文

第一作者： Zhenlong Yuan

作者： Zhenlong Yuan ¹ ; Jiakai Cao ¹ ; Zhaoxin Li ² ; Hao Jiang ¹ ; Zhaoqi Wang ¹ ;

作者机构： 1.Institute of Computing Technology, Chinese Academy of Sciences

2.Agricultural Information Institute, Chinese Academy of Agricultural Sciences

会议名称： AAAI Conference on Artificial Intelligence

主办单位：

页码： 6871-6880

摘要： In this paper, we introduce Segmentation-Driven Deformation Multi-View Stereo (SD-MVS), a method that can effectively tackle challenges in 3D reconstruction of textureless areas. We are the first to adopt the Segment Anything Model (SAM) to distinguish semantic instances in scenes and further leverage these constraints for pixelwise patch deformation on both matching cost and propagation. Concurrently, we propose a unique refinement strategy that combines spherical coordinates and gradient descent on normals and pixelwise search interval on depths, significantly improving the completeness of reconstructed 3D model. Furthermore, we adopt the Expectation-Maximization (EM) algorithm to alternately optimize the aggregate matching cost and hyperparameters, effectively mitigating the problem of parameters being excessively dependent on empirical tuning. Evaluations on the ETH3D high-resolution multi-view stereo benchmark and the Tanks and Temples dataset demonstrate that our method can achieve state-of-the-art results with less time consumption.

分类号： tp18-53