When solving the one-dimensional inviscid Burgers equation the face estimation via the Finite Volume method can be performed explicitly with a Q-Learning method alternatively to the traditional Godunov method. The Q-learning algorithm can learn from generalizable experiences and obtain effective optimization in highly-dimensional search spaces. The proposed methodology focuses on devising adequate training, validation and testing databases. Its performance is compared against the Godunov method in both interpolating and extrapolating test studies. There is a discussion on suitable Q-learning parameters, input treatment and adequate reward functions. During testing it is used a correlation-based metric and the order of the discretization is verified via error decay. The results produced are slightly better than the ones obtained with the Godunov method but they were achieved without requiring mathematical assumptions and empirically. This hopefully is easily extensible to more complex PDEs and higher-order discretizations.