Reinforcement Learning of Active Aerodynamics in Wind Tunnel Environments
Please login to view abstract download link
I will discuss our recent progress on applying Reinforcement Learning (RL) to chaotic and intractable turbulent fluid flow systems for optimising the aerodynamic efficiency through flow control and accelerating the transition to Net Zero. Since the governing fluid-flow equations are intractable for simulated environments for realistic geometries and high Reynolds numbers turbulent regimes, we apply RL in real time/real environments by interacting with the wind tunnel environment, which consists of a car model, dynamic actuators for aerodynamic shaping, and wall mounted pressure sensors. From an algorithmic perspective, I will discuss challenges associated to partial observability, convergence, and delays. A thorough analysis will be presented using simpler configurations before deploying them in the wind-tunnel environment. At laminar and two-dimensional flow regimes, the performance of the RL under partial observability of the flow dynamics (pressure sensors on the body) is significantly degraded, limiting drag reduction by 50% compared to probes optimally located downstream of the body. A method integrating memory into the control architecture is proposed to improve performance in partially observable systems. By augmenting the input to the controller (neural network) with a time series of lagged observations from past actions and sensors, the dynamic controllers discovered using RL completely stabilise the vortex shedding using only surface mounted sensors. From a hardware perspective, I will discuss efficient implementation of RL algorithms for real-time training and preliminary results from wind tunnel experiments of RL for road vehicle drag reduction. These results are a first step towards realistic implementation of reinforcement learning for optimising partially observable and intractable flows. Acknowledgements: This work is supported by the UKRI AI for Net Zero grant (EP/Y005619/1).