The discretization of parametric partial or ordinary differential equations describing complex physical phenomena leads to high-dimensional problems, the solution of which requires a massive computational effort. To address this challenge, the development of low-cost surrogates in a data driven manner is the main focus of this talk. To incorporate prior knowledge as well as measurement uncertainties in the traditional neural networks, an efficient sparse Bayesian training algorithm is introduced. By fine tuning specially designed priors the proposed scheme automatically determines relevant neural connections and adapts accordingly in contrast to the classical gradient-like solution. Due to its flexibility, the new scheme is less prone to overfitting, and hence can be used to approximate both forward and inverse maps by use of a smaller data set. The optimal choice of the measurement data then can be easily achieved by maximizing the information gain. In this talk the new type of learning will be showcased on a high-dimensional stochastic partial differential equation describing the nonlinear mechanics problem. The neural network architecture is taken to be of hybrid type being able to describe both time and spatial dependency.