Yilin (Tony) Zhuang

Diffusion models have gained attention for their ability to represent complex distributions and incorporate uncertainty, making them ideal for robust predictions in the presence of noisy or incomplete data. In this study, we develop and enhance score-based diffusion models in field reconstruction tasks, where the goal is to estimate complete spatial fields from partial observations. We introduce a condition encoding approach to construct a tractable mapping between observed and unobserved regions using a learnable integration of sparse observations and interpolated fields as an inductive bias. With refined sensing representations and an unraveled temporal dimension, our method can handle arbitrary moving sensors and effectively reconstruct fields. Furthermore, we conduct a comprehensive benchmark of our approach against a deterministic interpolation-based method across various static and time-dependent PDEs. Our study attempts to addresses the gap in strong baselines for evaluating performance across varying sampling hyperparameters, noise levels, and conditioning methods. Our results show that diffusion models with cross-attention and the proposed conditional encoding generally outperform other methods under noisy conditions, although the deterministic method excels with noiseless data. Additionally, both the diffusion models and the deterministic method surpass the numerical approach in accuracy and computational cost for the steady problem. We also demonstrate the ability of the model to capture possible reconstructions and improve the accuracy of fused results in covariance-based correction tasks using ensemble sampling.

Recent advances in generative artificial intelligence have had a significant impact on diverse domains spanning computer vision, natural language processing, and drug discovery. This work extends the reach of generative models into physical problem domains, particularly addressing the efficient enforcement of physical laws and conditioning for forward and inverse problems involving partial differential equations (PDEs). Our work introduces two key contributions: firstly, we present an efficient approach to promote consistency with the underlying PDE. By incorporating discretized information into score-based generative models, our method generates samples closely aligned with the true data distribution, showcasing residuals comparable to data generated through conventional PDE solvers, significantly enhancing fidelity. Secondly, we showcase the potential and versatility of score-based generative models in various physics tasks, specifically highlighting surrogate modeling as well as probabilistic field reconstruction and inversion from sparse measurements. A robust foundation is laid by designing unconditional score-based generative models that utilize reversible probability flow ordinary differential equations. Leveraging conditional models that require minimal training, we illustrate their flexibility when combined with a frozen unconditional model. These conditional models generate PDE solutions by incorporating parameters, macroscopic quantities, or partial field measurements as guidance. The results illustrate the inherent flexibility of score-based generative models and explore the synergy between unconditional score-based generative models and the present physically-consistent sampling approach, emphasizing the power and flexibility in solving for and inverting physical fields governed by differential equations, and in other scientific machine learning tasks.

We present the development of a semi-supervised regression method using variational autoencoders (VAE) for soft sensing of process quality variables. Recently, use of VAEs was proposed for regression applications based on variational inference. In this work, We extend this approach of supervised VAEs for regression to make it learn from both labelled and unlabelled data leading to a semi-supervised VAE for regression (SSVAER) formulation. The probabilistic regressor resulting from the variational approach makes it possible to estimate the variance of the predictions simultaneously, which provides a means for online uncertainty quantification for soft sensors. We provide an extensive comparative study of SSVAER with previously proposed semi-supervised learning methods on two soft sensing benchmark problems using fixed-size datasets, where we vary the percentage of labelled data available for training. In these experiments, SSVAER achieves the lowest test errors in 11 of the 20 studied cases, compared to other methods where the second best method gets 4 lowest test errors out of the 20.

Integrating post-combustion carbon capture and storage (CCS) facilities into fossil fuel power plants is considered an important step for reaching global carbon emission reduction targets. When the number of gas analyzers in such CCS units is limited, variations in the load of the power plant pose a challenge to determine the gaseous CO2 concentration profile in the absorber. A dynamic hybrid model for estimating the carbon capture absorber’s gaseous CO2 concentration profile has been proposed in this study. The model is built using actual process data collected from a carbon capture pilot plant and it combines data-driven and reaction kinetic (mechanistic) modeling approaches to act as a soft sensor along the absorber column. A subset of the process data is used for training the data-driven models and for estimating the parameters of the mechanistic model respectively. Dimensionality reduction techniques are applied to the data-driven model to reduce the input size and hence the size of the dynamic model elements. The outputs of the two models are fused by comparing the computed covariance matrices. A particular challenge for this work is that the collected process data has missing spatial labels and temporal values for the CO2 concentration measurements. The presented models are obtained using an encoding and interpolation approach for the missing information. For comparison, an alternative approach based on semi-supervised learning has been implemented. The performance of the resulting soft sensors is verified by using process data from previously unseen operating conditions. The soft sensor based on the proposed hybrid model outperforms the soft sensor trained by semi-supervised autoencoder. Overall the results indicate that the proposed approach can estimate the CO2 concentration percentage with an average root mean squared error of 0.123.