With extraordinary advances in sensor technology, infrared and visible image fusion has been widely used in both military and civilian applications. In this paper, we propose a novel image fusion method based on decomposition and division based strategy. The proposed method improves the guided filter to better decompose images and restrict artifacts around image boundaries. Furthermore, because the quality of visible images is easily affected by low light conditions and noises, it is necessary to enhance the contrast of visible images to improve the visual quality before applying image fusion. Subsequently, we divide the infrared and visible image into several sub-images in vertical direction, because there is more similar image content in this direction such as the sky and land. Additionally, each sub-image is decomposed into a base layer and a detail layer. Different from previous methods, the fusion in our proposed method is executed by two different strategies, one takes the sub infrared base layer as the main image to get the fusion result, while the other one takes the sub visible base layer as the main image, and two different sub-fusion results are obtained. We also propose a new fusion strategy called gradient-brightness criterion to adaptively output the final fused image. As a result, the fused image preserves more details of visible image and clearer infrared objects at the same time, which is well suited for human visual perception. Experimental results indicate that our proposed method achieves a superior performance compared with previous fusion methods in both subjective and objective assessments. © 2021 Elsevier B.V.