Scratches are one of the most common defects in industrial manufacturing. Weak scratches in the industrial environment have an ambiguous edge, low contrast, large span, and unfixed shape, which bring difficulty for automatic defect detection. Recently, many existing visual inspection methods based on deep learning cannot completely and effectively inspect industrial weak scratches due to the lack of discriminative features and sufficient spatial detail. In this article, a novel DeepScratchNet is proposed for automatic weak scratch detection by aggregating rich multidimensional feature for scratch representation. To obtain rich features, a pretrained ResNet block as a feature extractor is proposed in this article. To highlight features of scratch and weaken the noise, an attention feature fusion block (AFB) is proposed, which densely fuses high-level semantic features with low-level detail features using dual-attention mechanism. Due to the long span and connectivity of the weak scratches, a context fusion block (CFB) is proposed to learn the complete context. To further improve the scratch segmentation performance, the auxiliary loss is integrated into the proposed network. The proposed DeepScratchNet outperforms the traditional and other state-of-the-art deep learning-based methods on three given real-world industrial data sets with mIoU over 0.8005, 0.812, and 0.9286. The experimental results demonstrate that DeepScratchNet achieves good generalization capabilities.