Multi-agent formation control with obstacles avoidance (MAFC-OA) is one of the attractive tasks of multi-agent cooperation. Although a number of algorithms can achieve formation control effectively, they ignore the nature structure feature of the graph formed by agents. Given this problem, a model, MAFC-OA, which is composed of observation attention network, action attention network and Multi-long short-term memory (Multi-LSTM) is proposed. With MAFC-OA, the agents can be trained to form the desired formation and avoid dynamic obstacles in the environments with restricted communication. Specifically, the above two attention networks not only incorporate the influence of the nearby agents’ observation and actions, but also enlarge the agents’ receptive field (communication range) through the chain propagation characteristics to promote cooperation among agents. Moreover, the Multi-LSTM allows the agents to take obstacles into consideration in the order of distance and to avoid the obstacles effectively. Simulations demonstrate that the agents can form the desired formation and avoid dynamic obstacles effectively.