Recently, the Reinforcement Learning (RL) approach is proven efficiently in several domains (e.g., Atari games, Dota 2 game, Go, Self-driving cars, Protein folding, ...), especially with theinvention of Deep Reinforcement Learning (DRL). However, RL algorithms involve mostly the learning process of one single agent. In the real world, complex systems normally consist of multiple agents. In these systems, a hidden phenomenon might be exposed by the interaction of several agents, not by only one agent. Applying RL algorithms, especially DRL, in Multi-Agent Systems (MAS) is a potential approach to help us to look insight into the world. In this paper, we present a traffic congestion model for the one-way multi-lane highways and experiment with four RL algorithms in this setting.