This paper presents a study to estimate the structural damage of reinforced concrete (RC) buildings using nonlinear dynamic analysis of full solid finite element models. Today, detailed information about the structural damage, such as the distribution of local failure and the fracture process, is crucial to the seismic risk assessment of critical infrastructure (e.g., nuclear power plants). However, the conventional structural calculations are performed using frame analysis based on beam and spring elements to reduce the computation time, and these simple models should be elaborated for the precise risk assessment. In this study, the full solid finite element model of RC buildings is employed to show the distribution of local failure and the fracture process. In addition, the analysis procedure is based on the open-source parallel finite element method (FEM) code FrontISTR to improve the computational efficiency. Structural damage of RC buildings is caused by cracks in concrete and yielding of steel, exhibiting the nonlinear behaviour. In this study, a non-orthogonal multi-directional smeared crack model was employed as the material constitutive law to express the material nonlinearity of concrete. This model accepts multiple cracks in an element and selects the appropriate crack (active crack) from them at each time increment to simulate complicated crack failure under cyclic loading. However, the active crack procedure can cause unusual impact and large acceleration when switching active crack dynamically, such as seismic response. Therefore, the smeared crack model was modified to prevent unusual impact and achieve appropriate three-dimensional seismic analysis considering multiple cracks. Finally, the applicability of the proposed analysis method was examined through the nonlinear dynamic analysis of a shaking table test on a six-story RC wall frame building. The numerical results showed good agreement with experimental results in terms of response acceleration, displacement and observed fracture process. Therefore, it is concluded that the proposed analysis method can be applied to the seismic damage estimation of RC buildings in the future.