Let us machine learn!

Machine-learning-based turbulence generator (MLTG) (available on GitHub)

1. • The combination of a convolutional neural network-based bottleneck model and multi-layer perceptron provides an inflow condition of high-fidelity turbulent flow simulations instead of numerical driver or white Gaussian noise-based inflow.

   • x6000 acceleration with TESLA K40 GPU compared to a conventional driver.  

   • Reference: K. Fukami, Y. Nabae, K. Kawai, K. Fukagata, “Synthetic turbulent inflow generator using machine learning,” Physical Review Fluids, 4 (064603), 2019 (preprint: arXiv:1806.08903 [physics.flu-dyn]), [Animation1] [Animation2]

Super-resolution analysis with hybrid downsampled skip-connection/multi-scale model (hDSC/MS model) (available on UCLA Taira Lab)

1. • A customized CNN supported by the multi-size filters and skip connections achieves a reasonable spatial reconstruction of fluid flow data from extremely coarse low-resolution data. 

   • Reference 1: K. Fukami, K. Fukagata, K. Taira, “Super-resolution reconstruction of turbulent flows with machine learning,” Journal of Fluid Mechanics, 870, 106-120, 2019 (preprint: arXiv:1811.11328 [physics.flu-dyn])

   • Reference 2: K. Fukami, K. Fukagata, K. Taira, “Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows,” Journal of Fluid Mechanics, 909, A9, 2021 (preprint, arXiv:2004.11566 [physics.flu-dyn])

   • Reference 3: K. Fukami, K. Fukagata, K. Taira, “Super-resolution analysis via machine learning: A survey for fluid flows,” Theoretical and Computational Fluid Dynamics, 37, 421--444 (invited), 2023 (preprint, arXiv:2301.10937 [physics.flu-dyn])

Standard convolutional neural network-based autoencoder (CNN-AE) (available on GitHub)

1. • A sample code for nonlinear CNN-AE-based compression of unsteady flows is provided.

   • Example data sets of laminar cylinder wake at Re = 100 are also given.

   • Reference: K. Fukagata, K. Fukami, “Compressing fluid flows with nonlinear machine learning: mode decomposition, latent modeling, and flow control,” in Review, 2025

Mode-decomposing convolutional neural network-based autoencoder (MD-CNN AE) (available on Keio University Fukagata Lab)

1. • A customized CNN-AE can extract and visualize “nonlinear machine learning mode" contained in fluid flow data sets.

   • For a laminar cylinder wake, AE modes contain several linear POD modes thanks to nonlinear superposition. 

   • Reference: T. Murata, K. Fukami, K. Fukagata, “Nonlinear mode decomposition with convolutional neural networks for fluid dynamics,” Journal of Fluid Mechanics, 882, A13, 2020 (preprint: arXiv:1906.04029 [physics.comp-ph])

Convolutional neural network-based autoencoder and long short-term memory based reduced-order modeling (CNNAE-LSTM ROM) (available on GitHub, laminar and turbulence)

1. • High-resolution data are low-dimensionalized into latent space with CNN-AE. The mapped vector is then predicted by using LSTM, which enables us to allow the construction of ROM since the predicted low-dimensionalized vector can be reversed on the original domain with the CNN decoder.

   • We have investigated the applicability to unsteady laminar wakes and wall-bounded turbulence.

   • Reference 1: K. Hasegawa, K. Fukami, T. Murata, K. Fukagata, “Machine-learning-based reduced-order modeling for unsteady fluid flows with various bluff bodies,” Theoretical and Computational Fluid Dynamics, 34 (4), 367--383 (invited),  2020  (preprint, arXiv:2003.07548 [physics.flu-dyn])

   • Reference 2: K. Hasegawa, K. Fukami, T. Murata, K. Fukagata, “CNN-LSTM based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers,” Fluid Dynamics Research, 52, 065501, 2020

   • Reference 3: T. Nakamura, K. Fukami, K. Hasegawa, Y. Nabae, K. Fukagata, “Convolutional neural network and long short-term memory based reduced order surrogate for minimal turbulent channel flow," Physics of Fluids, 33, 025116, 2021 (preprint, arXiv:2010.13351 [physics.flu-dyn])

Hierarchical convolutional neural network-based autoencoder (available on Keio University Fukagata Lab)

1. • A customized CNN-AE based on a transfer learning idea is able to achieve a much more efficient low-dimensionalization compared to linear PCA and conventional AEs for both laminar and turbulent flows. 

   • Reference: K. Fukami, T. Nakamura, K. Fukagata, “Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data,” Physics of Fluids, 32, 095110, 2020 (preprint, arXiv:2006.06977 [physics.comp-ph])

Interpretable AI with probabilistic neural network (PNN) (available on GitHub)

1. • A fully-connected network trained with maximization of log-likelihood can tell us a confidence interval of its estimation in addition to the ML estimation.

   • Demonstrated with several flow data sets.

   • Reference: R. Maulik, K. Fukami, N. Ramachandra, K. Fukagata, K. Taira, “Probabilistic neural networks for fluid flow surrogate modeling and data recovery,” Physical Review Fluids, 5 (104401), 2020 (preprint, arXiv:2005.04271 [physics.flu-dyn])

Interpretable reduced-order modeling with sparse identification of nonlinear dynamics (SINDy) for low-dimensionalized flow representations (available on GitHub)

1. • Temporal evolution of low-dimensional form of flows (e.g., CNN-AE modes, temporal coefficients of linear models, etc...) is represented as a form of ODE with the support of SINDy.

   • The choice of regression function is also carefully examined.

   • Reference: K. Fukami, T. Murata, K. Zhang, K. Fukagata, “Sparse identification of nonlinear dynamics with low-dimensionalized flow representations," Journal of Fluid Mechanics, 926, A10, 2021 (preprint, arXiv:2010.12177 [physics.flu-dyn])

Grad-CAM for visualization of internal procedure of neural networks (available on GitHub)

1. • Grad-CAM is able to identify the `reason' of machine-learning-based estimation by taking a gradient of outputs at hidden layers.

   • The applicability of Grad-CAM for fluid flow regressions is examined. Both single and multi-dimension output problems are considered.

   • Reference: M. Morimoto, K. Fukami, K. Zhang, K. Fukagata, “Generalization techniques of neural networks for fluid flow estimation," Neural Computing and Applications, 34, 3647-3669, 2022 (preprint, arXiv:2011.11911 [physics.flu-dyn])

Convolutional neural network-based robust flow reconstruction from moving sensors assisted with Voronoi tessellation (available on GitHub)

1. • Sparse sensor measurements are projected onto Voronoi tessellation before being fed into a convolutional neural network.

   • This pre-processing with CNN can handle arbitrary numbers of sensors and moving sensors with a single machine learning model.

   • Reference: K. Fukami, R. Maulik, N. Ramachandra, K. Fukagata, K. Taira, “Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning," Nature Machine Intelligence, 3, 945-951, 2021 (preprint, arXiv:2101.00554 [physics.flu-dyn])

Convolutional neural network-based fluid flow modeling with the assistance of supplemental scalar inputs (available on GitHub)

1. • Focusing on the application of CNN for fluid flow analyses including reduced-order modeling and metamodeling, the influence of supplemental scalars such as Reynolds number is examined.  

   • Sample codes of scalar-input-assisted CNN-AE and CNN-MLP are available.

   • Reference: M. Morimoto, K. Fukami, K. Zhang, A. G. Nair, K. Fukagata, “Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low dimensionalization," Theoretical and Computational Fluid Dynamics, 35 (5), 633-658, 2021 (preprint, arXiv:2101.02535 [physics.flu-dyn])

ML-PIV: machine learning-based experimental velocity estimator from particle image (available on GitHub)

1. • Autoencoder-style convolutional neural network is used to estimate a velocity field from experimental particle images.  

   • Codes for the automatic generation of artificial particle image from DNS data are available.

   • Reference: M. Morimoto, K. Fukami, K. Fukagata, “Experimental velocity data estimation for imperfect particle images using machine learning,” Physics of Fluids, 33, 087121, 2021 

2D-3D CNN: volumetric data reconstruction from sectional fluid flow data (available on GitHub)

1. • Coupling of two- and three-dimensional convolutional neural networks is performed to reconstruct a three-dimensional fluid flow realization from two-dimensional sections. 

   • Towards extra data saving, the input two-dimensional sections can also be replaced with low-resolution images by combining with super-resolution analysis.

   • A code for generating “adaptive-sampled" low-resolution fluid flow snapshots is also available. 

   • Reference: M. Matsuo, K. Fukami, T. Nakamura, M. Morimoto, K. Fukagata, “Reconstructing three-dimensional bluff body wake from sectional flow fields with convolutional neural networks," SN Computer Science, 5, 306, 2024 (preprint, arXiv:2103.09020 [physics.flu-dyn])

Observable-augmented autoencoder for physically-interpretable manifold identification (available on GitHub)

1. • While nonlinear autoencoder-based fluid flow compression is powerful, the data distribution in a low-dimensional latent space has not often been cared.

   • Connecting the latent vectors and physical observables promotes physically-coherent manifold identification, demonstrated with both numerical (flows around an airfoil with extremely strong gust) and experimental (flows around a flat plate with transient gust encounter) aerodynamic data sets. 

   • Reference 1: K. Fukami, K. Taira, “Grasping extreme aerodynamics on a low-dimensional manifold," Nature Communications, 14, 6480, 2023 (preprint, arXiv:2305.08024 [physics.flu-dyn])

   • Reference 2: K. Fukami, H. Nakao, K. Taira, “Data-driven transient lift attenuation for extreme vortex gust-airfoil interactions," Journal of Fluid Mechanics, 992, A17, 2024 (preprint, arXiv:2403.00263 [physics.flu-dyn])

Phase autoencoder for nonlinear dynamics (available on GitHub)

• A range of nonlinear dynamics can often be expressed with a single-phase variable, although it is challenging to identify them, especially when the equations of motion are not available. 

• Focusing on limit-cycle oscillators, this customized autoencoder can encode the phase variable from a given physical variable.

• Reference: K. Yawata, K. Fukami, K. Taira, H. Nakao, “Phase autoencoder for limit-cycle oscillators," Chaos, 34, 063111, 2024 (preprint, arXiv:2403.06992 [nlin.AO]) (Selected as an Editor's Pick.)

Library-learning-assisted Robust PCA for denoising extremely corrupted flow data (available on GitHub)

• Experimentally measured fluid flow data are often contaminated with extremely large, sparse noise.

• Combining robust principal component analysis and the concept of library learning, which is often used in dynamical system identification, the current technique is able to denoise extremely corrupted flow fields without requiring any training data.

• A sample code with an example data set of numerically simulated NACA0012 airfoil wake is available. 

• Reference: P. Koop, I. Scherl, K. Fukami, “Library-learning-assisted robust principal component analysis for denoising severely corrupted flow fields," in Review.