Mechanical integrity assessment of components often relies on deterministic analyses, involving the use of observations from mechanical experiments to calibrate a constitutive model representing the 'average' material response. This calibrated model is then utilized to evaluate the mechanical behavior of components, often through finite element analysis. The limitations of deterministic approaches in assessing the reliability and safety margins of structures are widely acknowledged. However, the shift towards adopting probabilistic assessment methods, where both the calibration of constitutive models and mechanical analysis must consider material variabilities and other influencing factors, poses significant challenges, especially regarding the substantial computational costs associated with such assessments. This study utilizes physics-informed neural networks (PINNs) to address the computational cost challenge. Our objective is to calibrate an advanced temperature-dependent viscoplastic constitutive model using data from cyclic mechanical experiments. We derive probability distribution functions (PDFs) for each model parameter based on the experimental data and subsequently employ these PDFs for mechanical analyses of given components. The complexity of the constitutive model formulation results in high computational costs, even when pursuing deterministic calibration. This cost increases further when applying Bayesian inverse calibration techniques to derive PDFs for model parameters. To mitigate these challenges, we employ PINNs (unsupervised training) to create a surrogate for the constitutive model. The use of this surrogate significantly reduces the computational overhead associated with numerous model evaluations during inverse calibration. Additionally, we develop a second PINNs model for the parametric mechanical analysis of components, providing the full-field stress-strain response for any given combination of constitutive model parameters. This enables efficient probabilistic mechanical analysis of components. The presentation particularly focuses on the challenges of training PINNs for such applications and emphasizes the need to investigate the root causes of training issues, highlighting the role of relevant domain-specific knowledge in addressing these problems.