Generative Diffusion Model-Based Deep Reinforcement Learning for Uplink Rate-Splitting Multiple Access in LEO Satellite Networks

Abstract

This work studies the joint transmit power control and receive beamforming in uplink rate splitting multiple access (RSMA)-based low earth orbit (LEO) satellite networks, using both generative diffusion model and proximal policy optimization (PPO) learning framework. In particular, using RSMA, interference is partially decoded and partially treated as noise, thereby improving the spectral efficiency, while the dynamics and uncertainty in LEO satellite networks would pose challenges to the real-time power control and receive beamforming optimization. First, a long-run sum data rate maximization problem is formulated, subject to the individual data rate requirement, and then the Markov decision process (MDP) is used to model it. Second, on the basis of MDP, a generative diffusion model-based proximal policy optimization (PPO) framework is proposed, where a denoising network is taken as the actor network in PPO to output the optimal continuous policy, thereby facilitating the hyperparameter tuning and improve the sample efficiency. Finally, experiments are conducted to show advantages of merging diffusion model into PPO, in terms of larger spectral efficiency, by comparing proposed framework with benchmarks.