We present a fast method to simulate metal powder bed fusion additive manufacturing (PBFAM) processes on the part-scale to predict quantities of interest (QOI), such as temperature evolution, microstructural phase composition, and residual stresses. The fundamental computational challenge for part-scale simulations lies not so much in the requirements of the spatial approximation but rather in the large number of time steps necessary to resolve the fast-moving laser path. In contrast to many existing approaches in the literature, our model resolves the actual scan path, enabling new and detailed insights into the scan-strategy-dependent evolution of QOIs, especially for complex geometries. Starting from a highly efficient thermal model \cite{Proell2023}, we present an efficient implementation to predict the thermally-induced microstructural composition and the residual stress distribution. An emphasis is placed on an implementation that best utilizes available hardware. We use well-established techniques for parallel evaluation on distributed, adaptively refined meshes to distribute the work among available CPUs. Appropriate single instruction multiple data (SIMD) techniques are utilized to significantly speed up the evaluation times. Applying SIMD techniques to constitutive equations and evolution laws requires careful analysis of the different conditional branches within the model equations. All vector accesses and updates are performed in a cache-efficient manner. A performance analysis demonstrates the high degree of optimization of the presented approach. This computational framework allows to perform a coupled thermal microstructure simulation with a consistently resolved laser beam path for the complete build process of the AM Bench 2022 cantilever specimen (312 layers, ~30 million spatial degrees of freedom, ~50 million time steps) with a time-to-solution below two days. For the coupled thermo-mechanical problem, we present new application-specific solution schemes that enable fast simulations on hundreds of layers.