Deep Q Network Based Beam Tracking for Mobile Millimeter Wave Communications

In this paper, we present a beam tracking algorithm based on the deep Q-network (DQN) for mobile millimeter-wave (mmWave) communications. The proposed algorithm determines the receive beam angle from the received signals without knowing the channel model and dynamics. It uses the received signals of the current and previous time slots to design the state and reward of the DQN. Our goal is to maximize the signal-to-noise ratio by the actions of the designed DQN. A significant computational complexity reduction is achieved since the receiver does not need to run complicated signal processing algorithms once the DQN is properly trained. Therefore a practical implementation of mmWave beam tracking with a very large number of antennas under harsh mobile environments becomes feasible. Through the extensive simulations, we verified the performance of the proposed algorithm and demonstrated robustness to the system uncertainty and low computational complexity in comparison with particle filter and the Q-learning.