In several applications, e.g., for unfitted discretizations, finite element integrals need to be computed with quadrature formulas on unstructured point clouds, i.e., points that do not originate from a tensor product of a one-dimensional quadrature formula. The resulting high arithmetic complexity makes the efficient implementation of high-order methods challenging because the usually preferred matrix-free operator evaluation draws many of its attractive properties from the structure of shape functions and quadrature formulas [2]. We propose design choices and performance optimizations for tensor-product shape functions and demonstrate the performance by means of model benchmarks and application examples. With the proposed developments, we achieve more than an order-of-magnitude speedup for the operator evaluation of a discontinuous Galerkin discretization compared to the equivalent algorithm implemented by a sparse matrix-vector product. In practice, the speedup gets larger because no matrices need to be assembled. The functionality is applicable to a multitude of applications and methods including overlapping domain methods, fluid-structure-interaction, and multi-phase flow. The code infrastructure developed for this contribution is freely available in the finite element library deal.II [1].