Reinforcement learning (RL) [1] has emerged as a promising approach to automating decision processes in CFD. It allows to improve the performance of the simulations, leading to more accurate and efficient solvers. We explore different optimization problems, from a RL point of view, to improve our CFD simulations. In particular, we mainly focus on exploring the application of RL techniques to optimize the polynomial order in the computational mesh, based on our original idea developed in [2]. Mesh adaptation plays a crucial role in improving the efficiency of numerical simulations by increasing the accuracy while reducing the cost. We provide a strategy for p-adaptation in high-order solvers, including insights into the main aspects of RL-based mesh adaptation: the formulation of appropriate state-reward structures and the interaction between the RL agent and the simulation environment. The proposed strategy does not require a high-fidelity solution during the training process and the formulation is general for any computational mesh and partial differential equation, solved in a discontinuous Galerkin solver. Finally, we test the proposed methodology with our high-order solver HORSES3D [3] in several problems of interest and typical CFD benchmarks and compare our approach with traditional adaptive mesh refinement strategies. [1] R.S. Sutton and A.G. Barto, Reinforcement learning: an introduction. 2nd Edition, Elsevier, 1999. [2] D. Huergo, G. Rubio and E. Ferrer, A reinforcement learning strategy for padaptation in high order solvers. arXiv:2306.08292 (preprint). [3] E. Ferrer, G. Rubio, G. Ntoukas, W. Laskowski, O.A. Mariño, S. Colombo, A. Mateo-Gabín, H. Marbona, F. Manrique de Lara, D. Huergo, J. Manzanero, A.M. Rueda-Ramírez, D.A. Kopriva and E. Valero, A high-order discontinuous Galerkin solver for flow simulations and multi-physics applications. Computer Physics Communications, Vol. 287, 2023.