An off-axis three-mirror detection system with a large field of view is designed in order to improve the space target detection capability. The optical system is a Cook-TMA with the focal length of 127mm, the F number of 2, the field angle of 25° × 25° and the spectral range of 400-700nm. The primary mirror and the tertiary mirror of the off-axis three mirror system are all designed by free form surfaces: the primary mirror is characterized by Zernike polynomial and the tertiary mirror is described by XY polynomial. At the same time, we analyze the related characteristics of Zernike polynomial and XY polynomial. The results show that the free form surfaces have great advantages in improving the field of view and the imaging quality of the off-axis optical system. The system has high energy concentration and good imaging quality, which can capture and track the target in a wide range is suitable for wide area target monitoring. © 2018 SPIE.