In this paper, we present a novel computer-aided diagnostic (CAD) system based on the Breast Imaging Reporting and Data System (BI-RADS) terminology scores of screening ultrasonography (US). The decision tree algorithm is adopted to analyze the BI-RADS information to differentiate between the malignant and benign breast tumors. Although many ultrasonography CAD systems have been developed for decades, there are still some problems in clinical practice. Previous CAD systems are opaque for clinicians and cannot process the ultrasound image from different ultrasound machines. This study proposes a novel CAD system utilizing BI-RADS scoring standard and Classification and Regression Tree (CART) algorithm to overcome the two problems. The original dataset consists of 1300 ultrasound breast images. Three well-experienced clinicians evaluated all of the images according to the BI-RADS feature scoring standard. Subsequently, each image could be transformed into a 25x1 vector. The CART algorithm was finally used to classify these vectors. In the experiments, we used the oversampling method to balance the number of malignant samples and benign samples. The 5-fold cross validation was employed to evaluate the performance of the system. The accuracy reached 94.58%, the specificity was 98.84%, the sensitivity was 90.80%, the positive predictive value (PPV) was 98.91% and the negative predictive value (NVP) was 90.56%. The experiment results show that the proposed system can obtain a sufficient performance in the breast diagnosis and can effectively recognize the benign breast tumors in BI-RADS 3.