We explore the use of an uncrewed aerial vehicle (UAV) flying on a circular path to offload mobile data from a ground base station (GBS) to enhance cellular network capacity. The UAV’s performance is constrained by battery life and energy-intensive radio frequency communications. To address this, we jointly optimize energy efficiency (EE) and spectrum efficiency (SE) by adjusting the UAV’s trajectory, speed, and minimum user throughput. The multi-objective optimization problem we propose is complex and non-convex, presenting substantial challenges in finding an optimal solution. We develop a tailored deep reinforcement learning (DRL) approach to address this specific problem. Simulations show that our method effectively balances EE and SE, enhancing UAV-based cellular offloading while maintaining robust performance, even in uncertain and dynamic conditions.