A mathematical model of the diffraction efficiency change with the ambient temperature for the double-layer harmonic diffractive elements (HDE) is presented, and its effects are analyzed in this paper. The double-layer HDE structure is investigated and the optimization procedure is based on the equation of diffraction efficiency of the double-layer HDEs. By selecting appropriate design wavelength, the average diffraction efficiency of the system is reaching 99% in working wavelength and working temperature, which improves the image contrast and the image quality significantly. A set of dual-wavelength infrared optical system is designed based on dual-wavelength 320x240 element cooled thermal IR focal plane arrays detector. By introducing double-layer HDEs and aspheric surfaces, the chromatic aberration and the off-axis aberration are well corrected and the system structure is simplified. The system working in the wave band of 3.7?4.8μm and 7.7?9.5μm and with the F number of 2 is consisted of 8 elements and has 100% cold shield efficiency. The image quality evaluating results show that the performance of the dual-wavelength infrared optical system is very well in temperature from-40C° to +60C°. ? 2019 SPIE.