Education, Science, Technology, Innovation and Life
Open Access
Sign In

A Frequency Decomposition and Gaussian-Based Enhancement Network for Infrared and Visible Image Fusion

Download as PDF

DOI: 10.23977/acss.2024.080708 | Downloads: 9 | Views: 111

Author(s)

Ting Wang 1, Hao Song 1, Xuanyu Liao 1, Chengjiang Zhou 1

Affiliation(s)

1 School of Information Science and Technology, Yunnan Normal University, Kunming, China

Corresponding Author

Chengjiang Zhou

ABSTRACT

The purpose of infrared image and visible image fusion is to preserve information in different modalities. In order to solve the redundancy of modal frequency domain information extraction and feature mapping, we propose a frequency decomposition and Gaussian-Based enhancement network for infrared and visible image fusion. Firstly, we design a frequency decomposition convolution, which divides the feature map to realize the independent modeling of different frequency information, so as to extract the deep-level features more accurately. In addition, we design enhancement module combined with Gaussian filter to enhance the feature expression and optimize the loss function. Finally, we introduce dual-discriminators to refine the differentiation of infrared and visible images, significantly enhancing global information expression and detail presentation in fused image. Experimental outcomes demonstrate that our fusion method can effectively integrate the dominant information of the two images. Notably, our method outperforms other advanced fusion algorithms by enhancing the performance of object detection tasks, particularly in terms of improving the accuracy of detecting cars and pedestrians.

KEYWORDS

Feature enhancement, Frequency decomposition, Image fusion, Generative adversarial network

CITE THIS PAPER

Ting Wang, Hao Song, Xuanyu Liao, Chengjiang Zhou, A Frequency Decomposition and Gaussian-Based Enhancement Network for Infrared and Visible Image Fusion. Advances in Computer, Signals and Systems (2024) Vol. 8: 63-70. DOI: http://dx.doi.org/10.23977/acss.2024.080708.

REFERENCES

[1] Hu Y, He J, Xu L. Infrared and visible image fusion based on multiscale decomposition with Gaussian and co-occurrence filters[C]//2021 4th International Conference on Pattern Recognition and Artificial Intelligence (PRAI). IEEE, 2021: 46-50. 
[2] Yang C, He Y, Sun C, et al. Infrared and visible image fusion based on QNSCT and Guided Filter[J]. Optik, 2022, 253: 168592. 
[3] Li L, Ma H, Jia Z, et al. A novel multiscale transform decomposition based multi-focus image fusion framework[J]. Multimedia Tools and Applications, 2021, 80: 12389-12409. [4] Tan J, Zhang T, Zhao L, et al. Multi-focus image fusion with geometrical sparse representation[J]. Signal Processing: Image Communication, 2021, 92: 116130. 
[5] Xing C, Wang M, Dong C, et al. Using Taylor expansion and convolutional sparse representation for image fusion [J]. Neurocomputing, 2020, 402: 437-455. 
[6] Li H, Wu X J, Kittler J. MDLatLRR: A novel decomposition method for infrared and visible image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4733-4746. 
[7] Bavirisetti D P, Xiao G, Liu G. Multi-sensor image fusion based on fourth order partial differential equations[C]//2017 20th International conference on information fusion (Fusion). IEEE, 2017: 1-9. 
[8] Fu Z, Wang X, Xu J, et al. Infrared and visible images fusion based on RPCA and NSCT[J]. Infrared Physics & Technology, 2016, 77: 114-123. 
[9] Chen J, Wu K, Cheng Z, et al. A saliency-based multiscale approach for infrared and visible image fusion[J]. Signal Processing, 2021, 182: 107936. 
[10] Yang Y, Zhang Y, Huang S, et al. Infrared and visible image fusion using visual saliency sparse representation and detail injection model[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1-15. 
[11] Li H, Wu X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.  
[12] Ram Prabhakar K, Sai Srikar V, Venkatesh Babu R. Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//Proceedings of the IEEE international conference on computer vision. 2017: 4714-4722. 
[13] Xu H, Ma J, Jiang J, et al. U2Fusion: A unified unsupervised image fusion network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1): 502-518. [14] Tang L, Yuan J, Ma J. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42. 
[15] Ma J, Yu W, Liang P, et al. FusionGAN: A generative adversarial network for infrared and visible image fusion[J]. Information fusion, 2019, 48: 11-26. 
[16] Ma J, Xu H, Jiang J, et al. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. 
[17] Ma J, Zhang H, Shao Z, et al. GANMcC: A generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1-14. 
[18] Song A, Duan H, Pei H, et al. Triple-discriminator generative adversarial network for infrared and visible image fusion[J]. Neurocomputing, 2022, 483: 183-194. 
[19] Zhang Y, Liu Y, Sun P, et al. IFCNN: A general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. 
[20] Li H, Xu T, Wu X J, et al. Lrrnet: A novel representation learning guided fusion network for infrared and visible images[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(9): 11040-11052. 
[21] Tang W, He F, Liu Y, et al. DATFuse: Infrared and visible image fusion via dual attention transformer[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(7): 3159-3172. 
[22] Wang D, Liu J, Liu R, et al. An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detection[J]. Information Fusion, 2023, 98: 101828.                                                                                                                                        [23] Xie H, Zhang Y, Qiu J, et al. Semantics lead all: Towards unified image registration and fusion from a semantic perspective[J]. Information Fusion, 2023, 98: 101835.

Downloads: 27517
Visits: 464749

Sponsors, Associates, and Links


All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.