Reinforcement Learning for a Cellular Internet of UAVs: Protocol Design, Trajectory Control, and Resource Management
ieee 鏈接： https://ieeexplore.ieee.org/document/9023933
Citation:
J. Hu, H. Zhang, L. Song, Z. Han and H. V. Poor, “Reinforcement Learning for a Cellular Internet of UAVs: Protocol Design, Trajectory Control, and Resource Management,” in IEEE Wireless Communications, vol. 27, no. 1, pp. 116-123, February 2020, doi: 10.1109/MWC.001.1900262.
UAVs are able to have enough onboard computation and memory capacities to perform reinforcement learning approaches, In the following, we discuss the possible implementations of reinforcement learning to solve the trajectory control and resource management problems.
Q-Learning for Trajectory Control:
無論是 sensing 還是 transmission 都是越近越好
When the UAV executes a sensing task, the successful sensing probability will be higher if it gets closer to the sensing target. On the other hand, the UAV will have a higher probability of transmitting the sensory data to the BS successfully if it approaches the BS.
Therefore, sensing and transmission are coupled with its trajectory. However, since accurate sensing and transmission models are hard to obtain in the complex and dynamic environment of the cellular Internet of UAVs, trajectory control is also challenging.
space , action , trajectory , reward 的設計
As Q-learning does not require models of the sensing and transmission, it is suitable for the trajectory control problem. For applying Q-learning, the flight space can be abstracted into a finite set of discrete spatial points, and the trajectory can be considered as a path through these spatial points. The state of each UAV is its location, and the action is its trajectory in each cycle. Since the objective of each UAV is to send valid sensory data to the BS, the reward is 1 if the valid sensory data is received successfully by the BS; otherwise, the reward is 0.