The massive spread of false information has brought about severe security-related problems to individuals and society. To debunk misinformation automatically, fact checking has become an important task that aims at retrieving evidence from external sources to verify the truthfulness of a given claim. As knowledge graph (KG) is a classic external source for retrieving relevant evidence. Previous methods typically check a claim by making inferences over it. Entity category information can be utilized to strengthen both the learning and verification process. However, this information was largely ignored in previous research. To make better use of the category information, in this paper, we propose a category-based framework for improving the performance of fact checking with KGs. We first learn prototypes for each category as their representatives, and then propose a prototype-based learning technique for effectively modeling the entity dependency in KG. We further develop a prototype matching technique to explore the category-level relations between head and tail entities for more robust verification. Experimental results on two benchmark datasets and a real-world dataset show that our framework can significantly improve the reasoning abilities of KG reasoning methods on Fact Checking task.