Solving differential problems using full order models (FOMs), like the finite element method, incurs prohibitively computational costs in real-time simulations and multi-query routines. Reduced order modeling aims at replacing FOMs with reduced order models (ROMs), that exhibit significantly reduced complexity while retaining the ability to capture the essential physical characteristics of the system. In this respect, the novel concept of the Latent Dynamics Problem (LDP) is introduced and the class of Latent Dynamics Models (LDMs), along with their specialized deep learning counterpart, Neural Latent Dynamics Models (NLDMs) is presented. NLDMs constitute a neural differential equations-based model architecture designed for continuous-time modeling. This architecture is embedded within a reduced order modeling framework, with the primary objective of capturing the latent, low-dimensional dynamics of high-dimensional dynamical systems. In a series of numerical experiments, the effectiveness of NLDMs in addressing challenging problems is demonstrated, particularly in the context of large-scale high-fidelity models governed by time-dependent parameterized PDEs. The results not only underscore the remarkable performance of NLDMs but also highlight their potential as a valuable tool for understanding and modeling complex dynamic systems.