Videos taken by hand-held camera easily contain both motion and jitter, which can result in a large number of false detections of moving object detection and achieve poor performance. In this paper, we propose a moving object detection algorithm adapted to videos from hand-held camera. The proposed algorithm uses the optical flow method to perform motion estimation and motion compensation on the videos. So the interferences caused by hand-held camera can be reduced. Then we establish background model to detect the moving object. The proposed algorithm is verified with videos from hand-held camera and compared with several state-of-the-art algorithms. Experimental results demonstrate that the proposed algorithm is effective for moving object detection in videos from hand-held camera. © 2018 SPIE.