Improving the fidelity of quantum gates is crucial for using quantum computers in practical applications. Quantum optimal control methods allow for the construction of control pulses to induce desired transformations of quantum states. The time evolution of closed systems is well understood, but achieving low error rates on actual quantum computers requires the incorporation of dissipative effects and interaction with the environment. Various models exist for such open systems, but do not provide a complete description of all environmental effects on a quantum system. The shortcomings of the resulting models limit the efficacy of the optimal control pulses to physical quantum computers. Data-driven discovery methods have successfully captured the dynamics of systems for which existing equations are incomplete. In addition, known physics can be incorporated into some of these approaches, leading to better stability and improved generalization to new data. Recent machine learning frameworks augment data-driven approaches with constraints that enforce the laws of thermodynamics. We propose a scientific machine learning approach that learns thermodynamically consistent master equations for quantum systems interacting with the environment. The generalization properties of this approach are illustrated, including their applicability to new control Hamiltonians not used during training. We demonstrate the application of this framework to experimental qubit data and discuss the integration of learned dynamics with quantum optimal control methods.