This work addresses the development of a physics-informed neural network (PINN) with a physics-based loss term derived from an external CFD solver such as OpenFOAM. The current work is an extension of the authors' previous work (Halder et al., arxiv, 2023) where we have discussed how discretized governing equations can be coupled with the Artificial Neural Network (ANN) and Long-Short-Term-Memory (LSTM) based neural network. The major difficulties in coupling PINN with external forward solvers arise from the inability to access the discretized form of the governing equation directly through the PINN solver. This poses a significant challenge to conventional automatic-differentiation-based derivative computation of physics-based loss-term with respect to the weight matrix and bias vectors in neural networks. Therefore, we propose a modification of the physics-based loss term to account for the residual arising from the external solver and to compute the derivative required for the optimization machinery. To overcome the problem arising from large dimensionality, the governing equations are cast in linear and nonlinear manifolds as demonstrated by Giovanni et al. (computer and fluids, 2018) and Romor et al. (jsc, 2023) and also implemented in ITHACA-FV (Giovanni et al., computer and fluids, 2018). The resulting reduced form of the equation is used as a residual for the physics-based loss term in the PINN. Although the main advantages of the discretized physics-based neural network are elaborated in (Halder et al., arxiv, 2023), the main objective of the current work is to couple the available numerical data with existing forward solvers and use them for several inverse and ill-posed problems. The current work offloads the task of residual computation, boundary, and initial condition to dedicated external forward solvers such as OpenFOAM, and therefore additional implementation of governing physics in the conventional PINN framework is not required anymore. The potential of our proposed methods and implementation details are investigated on several benchmark applications.