Interpreting the Basis Path Set in Neural Networks

doi:10.1007/s11424-020-0112-y

	Interpreting the Basis Path Set in Neural Networks
	Zhu Juanping 3; Meng Qi 1; Chen Wei 1; Ma Zhiming 2
刊名	JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY
	2021-01-12
页码	13
关键词	Basis path hierarchical algorithm independent path neural network substructure path
ISSN号	1009-6124
DOI	10.1007/s11424-020-0112-y
英文摘要	The G-SGD algorithm significantly outperforms the conventional SGD algorithm in ReLU neural networks by adopting the basis path set. However, how the inner mechanism of basis paths works remains mysterious, and the G-SGD algorithm that helps to find a basis path set is heuristic. This paper employs graph theory to investigate structure properties of basis paths in a more general and complicated neural network with unbalanced layers and edge-skipping. The hierarchical Algorithm HBPS is proposed to find a basis path set, by decomposing the complicated network into several independent and parallel substructures. The paper theoretically extends the study of basis paths and provides one methodology to find the basis path set in a more general neural network.
资助项目	National Nature Science Foundation of China[11601471] ; Project for Innovation Team (Cultivation) of Yunnan Province[202005AE160006] ; Key Project of Yunnan Provincial Science and Technology Department and Yunnan University[2018FY001014]
WOS研究方向	Mathematics
语种	英语
出版者	SPRINGER HEIDELBERG
WOS记录号	WOS:000608140200033
内容类型	期刊论文
源URL	[http://ir.amss.ac.cn/handle/2S8OKBNM/58043]
专题	中国科学院数学与系统科学研究院
通讯作者	Zhu Juanping
作者单位	1.Microsoft Res Asia, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China 3.Yunnan Univ, Dept Math, Kunming 650500, Yunnan, Peoples R China
推荐引用方式 GB/T 7714	Zhu Juanping,Meng Qi,Chen Wei,et al. Interpreting the Basis Path Set in Neural Networks[J]. JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,2021:13.
APA	Zhu Juanping,Meng Qi,Chen Wei,&Ma Zhiming.(2021).Interpreting the Basis Path Set in Neural Networks.JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,13.
MLA	Zhu Juanping,et al."Interpreting the Basis Path Set in Neural Networks".JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY (2021):13.