热像仪因其能够通过检测物体表面热辐射而产生温度图像,近年来被广泛应用于多种工业场合。然而,由于热像图缺少直观的几何信息,当物体表面温度相近时,难以通过人眼分辨物体特征差异。为了解决这个问题,提出了一种结合可见光几何信息与热像图温度信息的三维模型重建方法。首先使用自制标定板进行可见光相机与热像仪的校正;随后,通过运动恢复结构与多视图立体匹配完成相机位姿和深度图的估计,并使用结合阴影与几何数据项的优化算法对深度图进一步优化;最终,借助相机参数和校正信息,利用互信息实现可见光图像与热像图之间的配准,结合配准结果和深度图实现了三维温度模型的重建。
Abstract
Thermal cameras, which can detect thermal emissions and produce images of radiation, are widely used in various industrial applications in recent years. However, without intuitive geometric information, it is difficult to identify objects on thermal images when adjacent sampling areas have the same temperature. To solve the above problem, a method to reconstruct a 3D thermal model by fusing visual geometry and temperature information is proposed. Firstly, a calibration board is used to calibrate the visible camera and thermal imager. Then, based on camera poses acquired by structure from motion, semi-global matching is used to obtain a coarse estimate of depth maps which are optimized by a refinement method combining geometry and shading term. Finally, with the help of camera parameters and calibration information, mutual information is used to achieve the registration between the visible and the thermal image, and the 3D temperature model is reconstructed through the depth maps.
关键词
计量学;温度模型;热像图;视觉几何;三维重建 /
互信息
Key words
metrology /
temperature model /
thermal images /
visual geometry /
3D reconstruction /
mutual information
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
［1］朱丹丹, 王斌, 杨奕, 等. Contourlet变换和遗传算法相结合的沥青红外图像增强方法［J］. 计量学报, 2019, 40(1): 25-30.
Zhu D D, Wang B, Yang Y, et al. Asphalt Infrared Image Enhancement of Combining Contourlet Transform with Genetic Algorithm［J］. Acta Metrologica Sinica, 2019, 40(01): 25-30.
［2］高一凡, 蔡静, 张学聪, 等. 基于NETD的红外热像仪图像预处理方法研究［J］. 计量学报, 2019, 40(6): 1020-1024.
Gao Y F, Cai J, Zhang X C, et al. Research on Image Processing Method of Infrared Camera Based on NETD［J］. Acta Metrologica Sinica, 2019, 40(6): 1020-1024.
［3］曲岩,宦克为,安保林, 等. 主动式双波长红外激光测温标定实验研究［J］. 计量学报, 2021, 42(2): 137-143.
Qu Y, Huan K W, An B L, et al. Study on active dual-wavelength infrared laser thermometry calibration experiments［J］. Acta Metrologica Sinica, 2021, 42(2): 137-143.
［4］Truong T P, Yamaguchi M, Mori S, et al. Registration of RGB and Thermal Point Clouds Generated by Structure From Motion［C］//IEEE Computer Society. 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). Venice, Italy, 2017: 419-427.
［5］Maset E, Fusiello A, Crosilla F, et al. Photogrammetric 3D building reconstruction from thermal images［C］//Remote Sensing and Spatial Information Sciences. International Conference on Unmanned Aerial Vehicles in Geomatics. Bonn, Germany, 2017: 25.
［6］Yang M D, Su T C, Lin H Y. Fusion of Infrared Thermal Image and Visible Image for 3D Thermal Model Reconstruction Using Smartphone Sensors［J］. Sensors, 2018, 18(7): 2003.
［7］Yamaguchi M, Truong T P, Mori S, et al. Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry［J］. IEICE TRANSACTIONS on Information and Systems, 2018, 101(5): 1296-1307.
［8］Matsumoto K, Nakagawa W, Saito H, et al. AR Visualization of Thermal 3D Model by Hand-held Cameras［J］. Cell Structure & Function, 2015, 33(2):171-183.
［9］Newcombe R A, Izadi S, Hilliges O, et al. KinectFusion: Real-time dense surface mapping and tracking［C］//IEEE Computer Society. 2011 10th IEEE International Symposium on Mixed and Augmented Reality. Basel, Switzerland, 2011: 127-136.
［10］Zhang Z Y. Flexible camera calibration by viewing a plane from unknown orientations［C］//IEEE. Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece, 1999: 666-673.
［11］Fuhrmann S, Langguth F, Goesele M. Mve-a multi-view reconstruction environment［C］//Citeseer, Eurographics Workshop on Graphics & Cultural Heritage. Darmstadt, Germany. 2014: 11-18.
［12］Lowe D G. Distinctive image features from scale-invariant keypoints［J］. International journal of computer vision, 2004, 60(2): 91-110.
［13］Bay H, Tuytelaars T, Van Gool L. Surf: Speeded up robust features［C］//Springer. European conference on computer vision. Graz, Austria, 2006: 404-417.
［14］Luong Q, Faugeras O. The fundamental matrix: Theory, algorithms, and stability analysis［J］. International journal of computer vision, 1996, 17(1): 43-75.
［15］Longuet-Higgins H C. A computer algorithm for reconstructing a scene from two projections［M］. San Francisco: Morgan Kaufmann, 1987: 61-62.
［16］Kneip L, Scaramuzza D, Siegwart R. A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation［C］//IEEE. The 24th IEEE Conference on Computer Vision and Pattern Recognition. Colorado, USA, 2011: 2969-2976.
［17］Triggs B, Mclauchlan P F, Hartley R I, et al. Bundle adjustment—a modern synthesis［C］//Springer. International workshop on vision algorithms. Corfu, Greece, 1999: 298-372.
［18］Hirschmuller H. Stereo Vision in Structured Environments by Consistent Semi-Global Matching［C］//IEEE. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR06). New York, USA, 2006: 2386-2393.
［19］Humenberger M, Engelke T, Kubinger W. A census-based stereo vision algorithm using modified Semi-Global Matching and plane fitting to improve matching quality［C］/IEEE. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, USA, 2010: 77-84.
［20］Zollhfer M, Dai A, Innmann M, et al. Shading-based refinement on volumetric signed distance functions［J］. ACM Transactions on Graphics, 2015, 34(4): 1-14.
［21］Langguth F, Sunkavalli K, Hadap S, et al. Shading-Aware Multi-view Stereo［C］//Springer. Computer Vision-ECCV 2016. Amsterdam, The Netherlands, 2016: 469-485.
［22］Edwin H. The retinex theory of color vision［J］. Entific American, 1978, 237(6): 108-128.
［23］Studholme C, Hill D, Hawkes D J. An Overlap Invariant Entropy Measure of 3D Medical Image Alignment［J］. Pattern Recognition, 1999, 32(1): 71-86.
［24］Fischler M A, Bolles R C. Random Sample Consensus: A Paradigm for Model Fitting with Applications To Image Analysis and Automated Cartography［J］. Communications of the ACM, 1981, 24(6): 381-395.
基金
浙江省自然科学基金(LY19F010007)
PDF(4034 KB)
