Abstract: A reinforcement learning agent model with intention tracking has been proposed to overcome the lagging in action in dynamic confrontation multi-agent systems (MAS) environment. The information of opponents and that of the environment have been treated differently. Based on reinforcement learning, the paper pays more attention on the intention tracking of the opponents. The intention tracking theory of Tambe have been improved, and opponent models and group-opponent models have been set up to track opponent intentions for forecasting the opponent's targets and revising agent-self's actions. Simulations have provided experimental results proving that agents with this model are more autonomic and adaptive.