The velocity gradient dynamics governs the physics of various nonlinear turbulent mechanisms like intermittency, scalar mixing and energy cascading etc. Therefore, developing simple dynamical models of velocity gradient tensors has been a topic of interest. Modeling of the pressure Hessian tensor and the viscous Laplacian process is required to close the velocity gradient evolution equation. Considering the highly nonlinear behavior of the pressure Hessian tensor, recently deep-learning based models have been proposed. Though these models show encouraging predictions, there is still a wide scope of improvement, especially in terms of the statistics of alignment between various tensors. Unlike any other previous approach, we propose to model the influence of the pressure Hessian tensor on the dynamics of the velocity gradient tensor, a combination of two different deep neural networks (DNN). The first DNN focuses on the alignment aspects of the pressure Hessian tensors, whereas a second DNN is trained to model the magnitude of the tensor. The input parameters for these networks are chosen to be the appropriately normalized invariants of the local velocity gradient tensor. The neural network parameters are then optimized using custom loss functions which are motivated by the physical nature of the pressure Hessian tensor observed in various direct numerical simulation studies. The predictions from both the neural networks are then integrated in the dynamical equation of the velocity gradient tensor. Indeed, the current model shows significant improvements over the existing models like the restricted Euler model, the homogenized Euler model and even the recent fluid deformation closure model.