Abstract：Thermal cameras, which can detect thermal emissions and produce images of radiation, are widely used in various industrial applications in recent years. However, without intuitive geometric information, it is difficult to identify objects on thermal images when adjacent sampling areas have the same temperature. To solve the above problem, a method to reconstruct a 3D thermal model by fusing visual geometry and temperature information is proposed. Firstly, a calibration board is used to calibrate the visible camera and thermal imager. Then, based on camera poses acquired by structure from motion, semi-global matching is used to obtain a coarse estimate of depth maps which are optimized by a refinement method combining geometry and shading term. Finally, with the help of camera parameters and calibration information, mutual information is used to achieve the registration between the visible and the thermal image, and the 3D temperature model is reconstructed through the depth maps.
张远辉,徐栢锐,朱俊江,孙坚. 可见光与热像融合的三维温度模型重建[J]. 计量学报, 2022, 43(2): 256-263.
ZHANG Yuan-hui,XU Bai-rui,ZHU Jun-jiang,SUN Jian. 3D Temperature Model Reconstruction Based on Fusion of Visible and Thermal Images. Acta Metrologica Sinica, 2022, 43(2): 256-263.
［1］朱丹丹, 王斌, 杨奕, 等. Contourlet变换和遗传算法相结合的沥青红外图像增强方法［J］. 计量学报, 2019, 40(1): 25-30.
Zhu D D, Wang B, Yang Y, et al. Asphalt Infrared Image Enhancement of Combining Contourlet Transform with Genetic Algorithm［J］. Acta Metrologica Sinica, 2019, 40(01): 25-30.
［2］高一凡, 蔡静, 张学聪, 等. 基于NETD的红外热像仪图像预处理方法研究［J］. 计量学报, 2019, 40(6): 1020-1024.
Gao Y F, Cai J, Zhang X C, et al. Research on Image Processing Method of Infrared Camera Based on NETD［J］. Acta Metrologica Sinica, 2019, 40(6): 1020-1024.
［3］曲岩,宦克为,安保林, 等. 主动式双波长红外激光测温标定实验研究［J］. 计量学报, 2021, 42(2): 137-143.
Qu Y, Huan K W, An B L, et al. Study on active dual-wavelength infrared laser thermometry calibration experiments［J］. Acta Metrologica Sinica, 2021, 42(2): 137-143.
［4］Truong T P, Yamaguchi M, Mori S, et al. Registration of RGB and Thermal Point Clouds Generated by Structure From Motion［C］//IEEE Computer Society. 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). Venice, Italy, 2017: 419-427.
［5］Maset E, Fusiello A, Crosilla F, et al. Photogrammetric 3D building reconstruction from thermal images［C］//Remote Sensing and Spatial Information Sciences. International Conference on Unmanned Aerial Vehicles in Geomatics. Bonn, Germany, 2017: 25.
［6］Yang M D, Su T C, Lin H Y. Fusion of Infrared Thermal Image and Visible Image for 3D Thermal Model Reconstruction Using Smartphone Sensors［J］. Sensors, 2018, 18(7): 2003.
［7］Yamaguchi M, Truong T P, Mori S, et al. Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry［J］. IEICE TRANSACTIONS on Information and Systems, 2018, 101(5): 1296-1307.
［8］Matsumoto K, Nakagawa W, Saito H, et al. AR Visualization of Thermal 3D Model by Hand-held Cameras［J］. Cell Structure & Function, 2015, 33(2):171-183.
［9］Newcombe R A, Izadi S, Hilliges O, et al. KinectFusion: Real-time dense surface mapping and tracking［C］//IEEE Computer Society. 2011 10th IEEE International Symposium on Mixed and Augmented Reality. Basel, Switzerland, 2011: 127-136.
［10］Zhang Z Y. Flexible camera calibration by viewing a plane from unknown orientations［C］//IEEE. Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece, 1999: 666-673.
［11］Fuhrmann S, Langguth F, Goesele M. Mve-a multi-view reconstruction environment［C］//Citeseer, Eurographics Workshop on Graphics & Cultural Heritage. Darmstadt, Germany. 2014: 11-18.
［12］Lowe D G. Distinctive image features from scale-invariant keypoints［J］. International journal of computer vision, 2004, 60(2): 91-110.
［13］Bay H, Tuytelaars T, Van Gool L. Surf: Speeded up robust features［C］//Springer. European conference on computer vision. Graz, Austria, 2006: 404-417.
［14］Luong Q, Faugeras O. The fundamental matrix: Theory, algorithms, and stability analysis［J］. International journal of computer vision, 1996, 17(1): 43-75.
［15］Longuet-Higgins H C. A computer algorithm for reconstructing a scene from two projections［M］. San Francisco: Morgan Kaufmann, 1987: 61-62.
［16］Kneip L, Scaramuzza D, Siegwart R. A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation［C］//IEEE. The 24th IEEE Conference on Computer Vision and Pattern Recognition. Colorado, USA, 2011: 2969-2976.
［17］Triggs B, Mclauchlan P F, Hartley R I, et al. Bundle adjustment—a modern synthesis［C］//Springer. International workshop on vision algorithms. Corfu, Greece, 1999: 298-372.
［18］Hirschmuller H. Stereo Vision in Structured Environments by Consistent Semi-Global Matching［C］//IEEE. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR06). New York, USA, 2006: 2386-2393.
［19］Humenberger M, Engelke T, Kubinger W. A census-based stereo vision algorithm using modified Semi-Global Matching and plane fitting to improve matching quality［C］/IEEE. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, USA, 2010: 77-84.
［20］Zollhfer M, Dai A, Innmann M, et al. Shading-based refinement on volumetric signed distance functions［J］. ACM Transactions on Graphics, 2015, 34(4): 1-14.
［21］Langguth F, Sunkavalli K, Hadap S, et al. Shading-Aware Multi-view Stereo［C］//Springer. Computer Vision-ECCV 2016. Amsterdam, The Netherlands, 2016: 469-485.
［22］Edwin H. The retinex theory of color vision［J］. Entific American, 1978, 237(6): 108-128.
［23］Studholme C, Hill D, Hawkes D J. An Overlap Invariant Entropy Measure of 3D Medical Image Alignment［J］. Pattern Recognition, 1999, 32(1): 71-86.
［24］Fischler M A, Bolles R C. Random Sample Consensus: A Paradigm for Model Fitting with Applications To Image Analysis and Automated Cartography［J］. Communications of the ACM, 1981, 24(6): 381-395.
2022, Vol. 43 
