Multi-scale full spike pattern for semantic segmentation

doi:10.1016/j.neunet.2024.106330

CORC > 自动化研究所 > 中国科学院自动化研究所 > 数字内容技术与服务研究中心 > 听觉模型与认知计算

	Multi-scale full spike pattern for semantic segmentation
	Su, Qiaoyi 2,4; He, Weihua 3; Wei, Xiaobao 2; Xu, Bo 2,4; Li, Guoqi 1,2,4
刊名	NEURAL NETWORKS
	2024-08-01
卷号	176 页码:12
关键词	Spiking neural network Semantic segmentation Neuromorphic computing Deep neural network Energy efficiency Brain-inspired computing
ISSN号	0893-6080
DOI	10.1016/j.neunet.2024.106330
通讯作者	Li, Guoqi(guoqi.li@ia.ac.cn)
英文摘要	Spiking neural networks (SNNs), as the brain-inspired neural networks, encode information in spatio-temporal dynamics. They have the potential to serve as low-power alternatives to artificial neural networks (ANNs) due to their sparse and event-driven nature. However, existing SNN-based models for pixel-level semantic segmentation tasks suffer from poor performance and high memory overhead, failing to fully exploit the computational effectiveness and efficiency of SNNs. To address these challenges, we propose the multi-scale and full spike segmentation network (MFS-Seg), which is based on the deep direct trained SNN and represents the first attempt to train a deep SNN with surrogate gradients for semantic segmentation. Specifically, we design an efficient fully-spike residual block (EFS-Res) to alleviate representation issues caused by spiking noise on different channels. EFS-Res utilizes depthwise separable convolution to improve the distributions of spiking feature maps. The visualization shows that our model can effectively extract the edge features of segmented objects. Furthermore, it can significantly reduce the memory overhead and energy consumption of the network. In addition, we theoretically analyze and prove that EFS-Res can avoid the degradation problem based on block dynamical isometry theory. Experimental results on the Camvid dataset, the DDD17 dataset, and the DSEC-Semantic dataset show that our model achieves comparable performance to the mainstream UNet network with up to 31 x fewer parameters, while significantly reducing power consumption by over 13 x . Overall, our MFS-Seg model demonstrates promising results in terms of performance, memory efficiency, and energy consumption, showcasing the potential of deep SNNs for semantic segmentation tasks. Our code is available in https://github.com/BICLab/MFS-Seg.
资助项目	National Science Foundation for National Science and Technology Major Project[2020AAA0105802] ; Distinguished Young Scholars[62325603] ; National Natural Science Foundation of China[62236009] ; National Natural Science Foundation of China[U22A20103] ; National Natural Science Foundation of China[62441606] ; Beijing Natural Science Foundation for Distinguished Young Scholars[JQ21015]
WOS关键词	NEURAL-NETWORK ; VIDEO ; MODEL
WOS研究方向	Computer Science ; Neurosciences & Neurology
语种	英语
出版者	PERGAMON-ELSEVIER SCIENCE LTD
WOS记录号	WOS:001235699200001
资助机构	National Science Foundation for National Science and Technology Major Project ; Distinguished Young Scholars ; National Natural Science Foundation of China ; Beijing Natural Science Foundation for Distinguished Young Scholars
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/58481]
专题	数字内容技术与服务研究中心_听觉模型与认知计算
通讯作者	Li, Guoqi
作者单位	1.Chinese Acad Sci, Inst Automat, Key Lab Brain Cognit & Brain Inspired Intelligence, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 3.Tsinghua Univ, Dept Precis Instrument, Beijing 100084, Peoples R China 4.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Su, Qiaoyi,He, Weihua,Wei, Xiaobao,et al. Multi-scale full spike pattern for semantic segmentation[J]. NEURAL NETWORKS,2024,176:12.
APA	Su, Qiaoyi,He, Weihua,Wei, Xiaobao,Xu, Bo,&Li, Guoqi.(2024).Multi-scale full spike pattern for semantic segmentation.NEURAL NETWORKS,176,12.
MLA	Su, Qiaoyi,et al."Multi-scale full spike pattern for semantic segmentation".NEURAL NETWORKS 176(2024):12.