The ability of a service robot to analyze and infer user's intent is essential to provide friendly service for users. This paper describes a novel and practical gaze-based intention inference framework. Existing frameworks primarily establish the relationship of gaze points with objects, which is the lack of mining of implicit information contained in eye gaze and prediction of human's intention. In our framework, the user's gaze is tracked using a model-based method and analyzed using two levels of unsupervised learning algorithms. The first level is that clustering gaze points and matching them with objects to enable the robot to understand semantic information of gazed objects. The second level is correlation analysis of gazed objects with behaviors, according to the user's daily living habits and gazed objects, the service robot can speculate concrete and proper behaviors from implicit user's intent. The advantage of this framework is that more effective and intuitive human-robot interaction can be realized only with a consumer level depth sensor (Kinect) and a normal laptop, even for the disabled person.