Two-stage Point Cloud Reconstruction Based on Improved Attention Mechanism and Surface Differential Geometry
WANG Kai1,CHEN Hui1,CHEN Lianming1,HUANG Heping2,CHEN Xiaolin3
1. College of Automation Engineering, Shanghai University of Electric Power, Shanghai 200090, China
2. Zhengtai Instrument (Hangzhou) Co. Ltd, Hangzhou, Zhejiang 310052, China
3. Shanghai Chint Power System Co. Ltd., Shanghai 201614, China
Abstract：The straight-through structure of direct prediction of point clouds in a single stage will lead to relatively sparse point clouds, unclear details and uneven distribution of results. Therefore, a 2-stage dense point cloud reconstruction network based on improved attention mechanism and local surface differential geometry is proposed, which can realize phased prediction of dense point clouds with different resolutions and uniform distribution. Firstly, the coordinate attention mechanism is improved by embedding to improve the networks ability to perceive the direction of the target and capture the coordinate information in the input image. Then, the sparse point cloud of the target is predicted, and the dense connection features of the sparse point cloud are extracted to obtain the structural perception information describing the sparse point cloud. Finally, the local surface differential geometry is used to complete the point cloud expansion, and then generate a dense point cloud with uniform distribution. The experimental results show that compared with 3D-FEGNet, the proposed dense point cloud reconstruction network reduces by 25.4% and 25.1% on CD and EMD, respectively. The three-dimensional dimensional errors of point clouds obtained by experiments with real objects are all less than 1.5mm.
WU J, WANG Y, XUE T, et al. Marrnet: 3D shape reconstruction via 2. 5D sketches ［C］//Advances in neural information processing systems(NIPS), NY, USA, 2017.
［6］
SUN X, LIAN Z. EasyMesh: An Efficient Method to Reconstruct 3D Mesh From a Single Image［J］. Computer Aided Geometric Design, 2020, 80: 101862.
ZHANG S H, WANG H L, CHEN Y X, et al. Object Detection Method Based on Feature Map Weighted Fusion Based on Deep Learning［J］. Acta Metrologica Sinica, 2020, 41(11): 1344-1351.
SONG J F, SUN B, PU Y Y, et al. Three-dimensional reconstruction of blade profile based on laser point cloud data［J］. Acta Metrologica Sinica, 2023, 44(2): 171-177.
［21］
CHANG A X, FUNKHOUSER T, GUIBAS L, et al. Shapenet: An Information-rich 3D Model Repository ［EB/OL］. https: //api. semanticscholar. org/CorpusID: 2554264, 2015-12-09.
LI H X, LI W L. Binocular Vision 3D Reconstruction Based on Deformable Convolution ［J］. Acta Metrologica Sinica, 2022, 43(6): 737-745.
［7］
WANG N, ZHANG Y, LI Z, et al. Pixel2mesh: Generating 3d Mesh Models From Single RGB Images ［C］//Proceedings of the European conference on computer vision (ECCV). 2018: 52-67.
［9］
FAN H, SU H, GUIBAS L J. A Point Set Generation Network for 3D Object Reconstruction From a Single Image ［C］//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 605-613.
［10］
MANDIKAL P, NAVANEET K L, AGARWAL M, et al. 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction From a Single Image［EB/OL］. https: //api. semanticscholar. org/CorpusID: 49905039, 2018-07-20.
［11］
ZHAO Y, JIAO J, LIN, et al. MANet: Multimodal Attention Network Based Point-view Fusion for 3D Shape Recognition ［C］//2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021: 134-141.
WANG S Q, ZHANG J Q, LI L Y, et al. Application of MVSNet in 3D Reconstruction of Spatial Targets［J］. Chinese Journal of Lasers, 2022, 49(23): 176-185.
ZHU Y Z, ZHANG Y P, FENG Q S. Single-view Color 3D Reconstruction Based on Deep Learning［J］. Laser & Optoelectronics Progress, 2021, 58(14): 207-215.
［14］
WANG E, SUN H, WANG B, et al. 3D-FEGNet: A Feature Enhanced Point Cloud Generation Network From a Single Image ［J］. IET Computer Vision, 2023, 17(1): 98-110.
［15］
WEN X, ZHOU J, LIU Y S, et al. 3D Shape Reconstruction From 2D Images With Disentangled Attribute Flow ［C］//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 3803-3813.
［17］
WANG P, LIU L, ZHANG H, et al. CGNet: A cascaded generative network for dense point cloud reconstruction from a single image［J］. Knowledge Based Systems, 2021, 223: 107057.
［18］
SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-scale Image Recognition［EB/OL］. https: //api. semanticscholar. org/CorpusI-D: 14124313, 2014-09-04.
［4］
XIE H, YAO H, SUN X, et al. Pix2vox: Context-aware 3D Reconstruction From Single and Multi-view Images ［C］//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 2690-2698.
HOU Q, ZHOU D, FENG J. Coordinate Attention for Efficient Mobile Network Design ［C］//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 13713-13722.
［22］
PING G, ESFAHANI M A, CHEN J, et al. Visual Enhancement of Single-view 3D Point Cloud Reconstruction［J］. Computers & graphics, 2022, 102: 112-119.
［8］
WEN C, ZHANG Y, LI Z, et al. Pixel2mesh++: Multi-view 3D Mesh Generation Via Deformation ［C］//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 1042-1051.
MANDIKAL P, RADHAKRIAHNAN V B. Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Network ［C］//2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2019: 1052-1060.
［20］
YI F W, WU S, HUANG H, et al. Patch-based Progressive 3D Point Set Upsampling ［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 5958-5967.
2024, Vol. 45 
