Simulating the mechanical response of advanced materials can be done more accurately using concurrent multiscale models than with single-scale simulations. However, the high computational costs stand in the way of the practical application of this approach. The costs originate from the microscale Finite Element (FE) models that must be solved at every macroscopic integration point. A plethora of surrogate modeling strategies attempt to alleviate this cost by learning to infer macroscopic stresses from macroscopic strains, completely replacing the microscale models. In this work, we introduce an alternative surrogate modeling strategy that allows for keeping the multiscale nature of the problem. Our surrogate provides all microscopic quantities, which are then homogenized to obtain the macroscopic quantities of interest. We achieve this for an elasto-plastic material by predicting the full-field microscopic strains using a graph neural network (GNN) while retaining the microscopic constitutive material model to obtain the stresses. This hybrid data-physics graph-based approach avoids the high dimensionality originating from predicting full-field responses while allowing non-locality to arise. In addition, this approach introduces beneficial inductive bias to the model by encoding microscopic geometrical features. By training the GNN on a variety of meshes, it learns to generalize to unseen meshes, allowing a single model to be used for a range of microstructures. The embedded microscopic constitutive model in the GNN implicitly tracks history-dependent variables and leads to improved accuracy and more efficient learning. As our model preserves the multiscale setup and computes all relevant quantities, it can be used interchangeably with an FE solver for any timestep. We demonstrate for several challenging scenarios that the surrogate can predict complex macroscopic stress-strain paths and be significantly faster than using the FE method for microscale simulations.