Attention Enhanced Reinforcement Learning for Multi agent Cooperation

doi:10.1109/TNNLS.2022.3146858

CORC > 自动化研究所 > 中国科学院自动化研究所 > 综合信息系统研究中心 > 飞行器智能技术

	Attention Enhanced Reinforcement Learning for Multi agent Cooperation
	Pu, Zhiqiang 2; Wang, Huimu 1,2; Liu, Zhen 2; Yi, Jianqiang 2; Wu, Shiguang 2
刊名	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
	2022-02-17
页码	15
关键词	Training Reinforcement learning Games Scalability Task analysis Standards Optimization Attention mechanism deep reinforcement learning (DRL) graph convolutional networks multi agent systems
ISSN号	2162-237X
DOI	10.1109/TNNLS.2022.3146858
通讯作者	Wang, Huimu(wanghuimu2018@ia.ac.cn)
英文摘要	In this article, a novel method, called attention enhanced reinforcement learning (AERL), is proposed to address issues including complex interaction, limited communication range, and time-varying communication topology for multi agent cooperation. AERL includes a communication enhanced network (CEN), a graph spatiotemporal long short-term memory network (GST-LSTM), and parameters sharing multi-pseudo critic proximal policy optimization (PS-MPC-PPO). Specifically, CEN based on graph attention mechanism is designed to enlarge the agents' communication range and to deal with complex interaction among the agents. GST-LSTM, which replaces the standard fully connected (FC) operator in LSTM with graph attention operator, is designed to capture the temporal dependence while maintaining the spatial structure learned by CEN. PS-MPC-PPO, which extends proximal policy optimization (PPO) in multi agent systems with parameters' sharing to scale to environments with a large number of agents in training, is designed with multi-pseudo critics to mitigate the bias problem in training and accelerate the convergence process. Simulation results for three groups of representative scenarios including formation control, group containment, and predator-prey games demonstrate the effectiveness and robustness of AERL.
资助项目	National Key Research and Development Program of China[2018AAA0102404] ; National Natural Science Foundation of China[62073323] ; National Natural Science Foundation of China[61806199] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27030403] ; External Cooperation Key Project of Chinese Academy Sciences[173211KYSB20200002]
WOS关键词	LEVEL ; GAME ; GO
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000761254200001
资助机构	National Key Research and Development Program of China ; National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Sciences ; External Cooperation Key Project of Chinese Academy Sciences
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/47921]
专题	综合信息系统研究中心_飞行器智能技术
通讯作者	Wang, Huimu
作者单位	1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Pu, Zhiqiang,Wang, Huimu,Liu, Zhen,et al. Attention Enhanced Reinforcement Learning for Multi agent Cooperation[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2022:15.
APA	Pu, Zhiqiang,Wang, Huimu,Liu, Zhen,Yi, Jianqiang,&Wu, Shiguang.(2022).Attention Enhanced Reinforcement Learning for Multi agent Cooperation.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,15.
MLA	Pu, Zhiqiang,et al."Attention Enhanced Reinforcement Learning for Multi agent Cooperation".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2022):15.