The concept of concurrent material and structure optimization constitutes a promising pathway to elucidate optimum microstructure configurations in multi-phase hierarchical systems such as plants, bones, and concrete. It is based on the split of the multiscale optimization problem into two nested sub-problems, one at the macroscale (structure) and the other at the microscales (material). In this presentation, we establish a novel thermodynamically consistent theoretical foundation for an efficient concurrent material and structure optimization framework that can tackle the computing challenge of optimizing elastoplastic multiphase hierarchical structures within the context of the finite element method. In particular, we reformulate the material optimization problem based on the maximum plastic dissipation principle such that it assumes the format of an elastoplastic constitutive law and can be efficiently solved via modified return mapping algorithms. We integrate continuum micromechanics-based estimates of homogenized stiffness and yield criterion, which enables material optimization via a series of inexpensive constraint optimization problems whose computational cost is independent of the number of hierarchical scales involved. To demonstrate the accuracy and robustness of our framework, we define new benchmark tests with several material scales that become computationally feasible for the first time. We contend that our formulation inherently expands to encompass multiscale optimization involving additional path-dependent effects like viscoelasticity or multiscale fracture and damage.