Phase retrieval occurs in many research areas. There are some classical phase retrieval methods such as hybrid input-output (HIO) and difference map (DM). However, phase retrieval results are sensitive to noise, and the reconstructed images always include artefacts. In this paper, we use the DM algorithm together with DNN to get better phase retrieval results. We train one deep neural network using amplitude images and phase images, respectively. First, using DM, we get initial reconstructed amplitude and phase results. Then, using DNN improves both amplitude and phase results. Finally, using the DM algorithm again improves the DNN results further. The numerical experimental results show that using DM gives better results than HIO, and using DNN improves phase information better than just using DNN to train for amplitude information alone. Compared with only using DNN improves amplitude methods, our method using DM plus DNN plus DM yields a better reconstruction performance for both amplitude and phase. © 2021 Informa UK Limited, trading as Taylor & Francis Group.