Hard-magnetic soft materials (HMSMs) belong to the class of active materials that undergo finite deformation and alter their material properties when subjected to magnetic loading. Owing to these remarkable characteristics, the HMSMs have shown promise in many applications such as soft robotics, soft actuators, phononic structures, and many more. In many applications, the HMSMs-based devices undergo time-dependent motions during their operation. However, there is a rare study on the dynamic modeling of hard-magnetic soft actuators. To this end, we develop a three-dimensional implicit dynamic finite element framework based on the nonlinear field theory of HMSMs for simulating the actuation response of the hard-magnetic soft actuators subjected to time-varying magnetic flux density. The neo-Hookean model of hyperelasticity in conjunction with the ideal hard-magnetic soft material model is used for characterizing the constitutive behaviour of HMSMs. Analytical expressions are provided for the stress tensor components and constitutive moduli. A selective reduced integration scheme is utilized for mitigating the volumetric locking developed due to nearly incompressible material behaviour. A standard eight-node hexahedron element is used for FE discretization. An in-house MATLAB code is developed for implementing the proposed dynamic finite element formulation. Using the developed finite element framework, the response of a homogeneously deforming hard-magnetic soft actuator under different time-varying magnetic loading i.e., DC, AC, and combined DC+AC loading, is simulated. The numerical results and the associated inferences can find their potential applications in addressing the challenges in the design and development of hard- magnetic material-based soft actuators exhibiting dynamic motion.