As modern systems are becoming more complex, their control strategy can no longer rely only on measurement information that usually comes from probes but also from mathematical models. Those systems models can lead to unbearable computation times due to their complexity, turning the control process non-viable, which leads to the implementation of surrogate models that enable to achieve of estimates within an acceptable time to make decisions. A control trained with a Deep Reinforcement Learning algorithm, using a Physics Informed Neural Network to obtain the temperature map on the following time step, replaces the need to run Direct Numerical Simulations. In this work, we considered a 1D heat conduction problem, in which temperature distribution feeds a control system to activate a heat source aiming to obtain a constant, previously defined, temperature value. With this approach, control training becomes much faster without the need to perform numerical simulations or laboratory measurements, as well the control is taken based on a Neural Network enabling its implementation on simple processors to edge computing.