A proposal of coupling deep reinforcement learning and high-fidelity fluid simulation in separation controls over an airfoil
Please login to view abstract download link
Recently, DBD plasma actuators (PA) have been actively studied as microflow control devices for suppressing separated flows around airfoils. Burst actuation, which is determined by the nondimensional burst frequency (F^+), has demonstrated superior separation control capability compared to continuous actuation. However, it is still difficult to determine the appropriate F^+ for varying flow conditions. Shimomura et al.[1] attempted to determine an appropriate burst driving method using the deep Q-learning network through the experimental study and discovered effective driving strategies. In experiments, the information obtained regarding the flow fields is limited. On the other hand, it is possible to obtain more information by utilizing numerical simulation. The objective of the present study is to propose a simulation framework coupling high-fidelity simulation using a supercomputer and Deep Reinforcement Learning (DRL). In the present study, we construct a simulation framework consisting of a DRL program and a CFD solver, which would be a new use case of supercomputers, as shown in Fig. 1. Inter-process communication is used to exchange information such as state, reward, and action between a supercomputer running the CFD solver and a machine running the DRL process. The DRL agent determines the optimal F^+ based on time series pressure data obtained from sensors on the airfoil surface. The reward is given according to the trailing edge pressure. Learning episodes are repeated many times to learn the control strategy. Figure 2 shows the results of episode 4 7 and 9. It shows that trailing edge separation is suppressed, although more learning is considered necessary to maintain it. The framework made it possible to obtain higher resolution of flow data with dynamic changing F^+ in each episode shown in Fig. 3.