MS161 - Unravelling Neural Networks with Structure-Preserving Computing
Keywords: chaos, differential equations, machine learning, mimetic methods, N-body systems, stability, structure preservation, turbulence models, neural networks
The understanding of processes and phenomena in science and engineering is radically transformed by machine learning. Many scientists and engineers embrace machine learning as an important tool. At the same time, obstacles and challenges are becoming apparent: most machine learning approaches require large amounts of data, but in many applications data is scarce. Furthermore, the performance and reliability of artificial neural networks âˆ’ the dominant type of â€˜learned machinesâ€™ âˆ’ is usually difficult to interpret. The ultimate goal of this mini-symposium is: to reveal how neural networks can be made more effective and efficient, and better understood, by incorporating mathematical and physical knowledge into their design. The direct goal of the mini-symposium is to contribute to the development of theory for mimetic neural networks for data-efficient and well-understood use in computational science and engineering. To achieve the above, it is necessary to have contributions from multiple disciplines. The mini-symposium will have expert speakers from mathematics, computer science, machine learning, physics and astronomy. From a mathematical point of view, there are many open questions. Overarching is the question: How can we optimally embed prior knowledge into neural networks? More specific questions are: How can we create interpretable machine-learning models? How can we further optimize training algorithms for neural networks? What are the nonlinear stability conditions and other requirements of neural networks? Answers to these questions are very important for among others fluid mechanics and astro-mechanics, with essential opportunities for cross-pollination and mutual benefits for both. Work by astronomers on the N-body problem demonstrates that neural networks can be used to make N-body computations, with NÂ»1, much more efficient. In fluid mechanics, machine learning methods for the analysis, modelling, and control of turbulent flows are currently developed to answer both fundamental and applied questions. The intrinsic chaotic behaviour of multi-body systems in astro-mechanics resembles the non-trivial statistical properties of turbulence in fluid mechanics. The mathematically inclined contributions to the mini-symposium will concentrate on fundamental properties of neural networks. The fluid mechanics and astro-mechanics contributions concentrate on specific challenges which serve as test cases for potentially more general strategies.