Characterization of heterogeneous multiscale hydraulic conductivity and identification of contaminant sources are routinely formulated as inverse problems. Numerical solution of such problems is nontrivial due to high dimensionality of the parameters space and high computational cost of multiple solves of a forward model. Cheap-to-compute surrogates of the latter, e.g., neural networks (NNs), might ameliorate this challenge but they are expensive to train. The acquisition of large amounts of training data from a physics-based model might be as expensive as the inverse problem’s solution. We address this issue by using multi-fidelity simulations to reduce the cost of data generation and using transfer learning to train a deep convolutional neural network (CNN) on these data. High- and low-fidelity images are generated by solving the forward multi-phase flow problem on fine and coarse meshes, respectively. The resulting multi-fidelity temporal snapshots of saturation maps are used to train a CNN in three phases. During the first phase, the training utilizes solely low-fidelity data. In the subsequent two steps, involving training of the output layer and fine-tuning of the overall network, the learning of the CNN features is finalized requiring only a relatively small number of high-fidelity data. We demonstrate that our approach provides an optimal balance between computational speed-up and predictive accuracy.