In a large-scale structure assembly, a highly accurate normal direction measurement for robotic drilling is required. However, the robustness and accuracy of the existing normal direction measurement method with a range sensor would dramatically decrease for a high-curvature surface, such as the surface edge, due to the low resolution of the range sensor. To this end, a high-resolution fringe projection sensor is proposed to improve the measurement robustness and accuracy. In this method, different from the control scheme design with a reconstructed three-dimensional point cloud in fringe projection profilometry, which involves a high computational complexity, a novel two-dimensional phase-map-based two-stage control scheme, including coarse and refined normal direction adjustments, is designed to improve the robustness and accuracy. The coarse normal direction adjustment uses a phase map to estimate the normal direction, which is used to synthesize a position-based control scheme, leading to high robustness; the refined normal direction adjustment uses a phase map to synthesize an image-based control scheme, resulting in high accuracy. The proposed method is validated by simulations and experiments, which indicate that the proposed method is effective.