Convolutional Neural Network (CNN) based object detection has achieved remarkable progress. However, most existing methods work on closed set assumption and can detect only objects of known classes. In real-world scenes, an image may contain unknown-class foreground objects that are unseen in training set but of potential interest, and open set object detection aims at detecting them as foreground, rather than rejecting them as background. A few methods have been proposed for this task, but they suffer from either low speed or unsatisfactory ability of unknown identification. In this paper, we propose a one-stage open set object detection method based on prototype learning. Benefiting from the compact distributions of known classes yielded by prototype learning, our method shows superior performance on identifying objects of both known and unknown classes from images in the open set scenario. It also inherits all advantages of YOLO v3 such as the high inference speed and the ability of multi-scale detection. To evaluate the performance of our method, we conduct experiments with both closed & open set settings, and especially assess the performance of unknown identification using recall and precision of the unknown class. The experimental results show that our method identifies unknown objects better while keeping the accuracy on known classes.