publications

2026

1. NeurIPS 2026

   

   Dynamic Quadtree Tokenization and Transformer for Adaptive Mesh PDE Forecasting

   Yilin Zhuang, Noah Zambrano, and Karthik Duraisamy

   In Advances in Neural Information Processing Systems (NeurIPS), 2026

   Abs arXiv Bib Code

   

   The quadratic attention cost of Vision Transformers (ViTs) forces a trade-off between spatial resolution and rollout horizon, particularly for fine-scale PDEs where shocks, reaction fronts, and material interfaces occupy small, evolving regions of the domain. Conventional neural surrogates also lack mechanisms to adapt resolution dynamically. We propose WAMRViT, a ViT that tokenizes inputs as balanced quadtrees using a wavelet-inspired refinement criterion, jointly encodes position and refinement level with 3D rotary positional embeddings, and regrids in cell space during inference for stable long-horizon rollouts. A multi-scale variant retains each leaf at its native source resolution and lets the model learn across resolution levels. Unlike fixed-budget adaptive-tokenization methods, WAMRViT imposes no predetermined token count and supports fully adaptive topology throughout autoregressive rollout. To our knowledge, it is the first machine-learning surrogate to natively tokenize multi-level Adaptive Mesh Refinement (AMR) data. On uniform-grid benchmarks, uniform-patch WAMRViT improves finest-level region-of-interest VRMSE over a finest-patch uniform ViT while using substantially fewer tokens. The multi-scale variant achieves the lowest first-step full-field RMSE and VRMSE on both benchmarks and improves rollout-averaged full-field and refined-region accuracy at long horizons. With parallelized regridding, its end-to-end rollout cost lies between finest-patch and approximately token-matched coarser-patch ViTs. On a complex AMR combustion problem whose finest features cannot be represented natively by the evaluated uniform-grid baselines, WAMRViT operates directly on adaptive cells and substantially reduces finest-level error at matched transformer capacity.

   @inproceedings{zhuang2026wamrvit,
     title = {Dynamic Quadtree Tokenization and Transformer for Adaptive Mesh {PDE} Forecasting},
     author = {Zhuang, Yilin and Zambrano, Noah and Duraisamy, Karthik},
     booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
     year = {2026},
     archiveprefix = {arXiv},
     primaryclass = {cs.CV},
     url = {https://arxiv.org/abs/2610.04044},
     keywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning},
   }

2. NeurIPS 2026

   

   Evolutionary Feature Engineering for Structured Data

   Ege Onur Taga, Yilin Zhuang, M. Emrullah Ildiz, and 4 more authors

   In Advances in Neural Information Processing Systems (NeurIPS), 2026

   Abs arXiv Bib Code

   

   Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data. EFE represents transformations as Python programs with a standardized fit/transform interface, allowing them to be inserted directly into existing machine learning pipelines. During evolution, candidate programs are refined using dataset context, summary statistics, and downstream performance feedback on validation set. We instantiate EFE in two settings. For time-series forecasting, EFE-Time learns invertible, dataset-specific normalizations that improve off-the-shelf time-series foundation models. It reduces forecasting errors (MASE, WQL, MAE) 3% or more when averaged across datasets and improvements are as much as 19% on the COVID-Deaths dataset. Notably, these improvements occur with recent TSFMs such as Chronos-2. For tabular prediction, EFE-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones, improving or matching existing LLM-based feature-engineering methods. We found EFE-Tab to be particularly effective on classical decision trees, where small sets of evolved features yield competitive accuracy while preserving interpretability. Overall, EFE demonstrates that LLM-based evolution can improve both accuracy and interpretability when automatically tackling structured data.

   @inproceedings{taga2026efe,
     title = {Evolutionary Feature Engineering for Structured Data},
     author = {Taga, Ege Onur and Zhuang, Yilin and Ildiz, M. Emrullah and Mol, Petros and Das, Abhimanyu and Duraisamy, Karthik and Oymak, Samet},
     booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
     year = {2026},
     archiveprefix = {arXiv},
     primaryclass = {cs.LG},
     url = {https://arxiv.org/abs/2607.01548},
     keywords = {Computer Science - Machine Learning},
   }

2025

1. 

   LaDCast: A Latent Diffusion Model for Medium-Range Ensemble Weather Forecasting

   Yilin Zhuang and Karthik Duraisamy

   2025

   Abs DOI arXiv Bib Code

   

   Accurate probabilistic weather forecasting demands both high accuracy and efficient uncertainty quantification, challenges that overburden both ensemble numerical weather prediction (NWP) and recent machine-learning methods. We introduce LaDCast, the first global latent-diffusion framework for medium-range ensemble forecasting, which generates hourly ensemble forecasts entirely in a learned latent space. An autoencoder compresses high-dimensional ERA5 reanalysis fields into a compact representation, and a transformer-based diffusion model produces sequential latent updates with arbitrary hour initialization. The model incorporates Geometric Rotary Position Embedding (GeoRoPE) to account for the Earth’s spherical geometry, a dual-stream attention mechanism for efficient conditioning, and sinusoidal temporal embeddings to capture seasonal patterns. LaDCast achieves deterministic and probabilistic skill close to that of the European Centre for Medium-Range Forecast IFS-ENS, without any explicit perturbations. Notably, LaDCast demonstrates superior performance in tracking rare extreme events such as cyclones, capturing their trajectories more accurately than established models. By operating in latent space, LaDCast reduces storage and compute by orders of magnitude, demonstrating a practical path toward forecasting at kilometer-scale resolution in real time. We open-source our code and models and provide the training and evaluation pipelines at: https://github.com/tonyzyl/ladcast.

   @misc{zhuang2025ladcastlatentdiffusionmodel,
     title = {LaDCast: A Latent Diffusion Model for Medium-Range Ensemble Weather Forecasting},
     author = {Zhuang, Yilin and Duraisamy, Karthik},
     year = {2025},
     archiveprefix = {arXiv},
     primaryclass = {cs.LG},
     url = {https://arxiv.org/abs/2506.09193},
     doi = {https://doi.org/10.48550/arXiv.2506.09193},
     keywords = {Computer Science - Machine Learning},
   }

2. CMAME

   

   Spatially-aware diffusion models with cross-attention for global field reconstruction with sparse observations

   Yilin Zhuang, Sibo Cheng, and Karthik Duraisamy

   Computer Methods in Applied Mechanics and Engineering, 2025

   Abs DOI arXiv Bib Code

   

   Diffusion models have gained attention for their ability to represent complex distributions and incorporate uncertainty, making them ideal for robust predictions in the presence of noisy or incomplete data. In this study, we develop and enhance score-based diffusion models in field reconstruction tasks, where the goal is to estimate complete spatial fields from partial observations. We introduce a condition encoding approach to construct a tractable mapping between observed and unobserved regions using a learnable integration of sparse observations and interpolated fields as an inductive bias. With refined sensing representations and an unraveled temporal dimension, our method can handle arbitrary moving sensors and effectively reconstruct fields. Furthermore, we conduct a comprehensive benchmark of our approach against a deterministic interpolation-based method across various static and time-dependent PDEs. Our study attempts to addresses the gap in strong baselines for evaluating performance across varying sampling hyperparameters, noise levels, and conditioning methods. Our results show that diffusion models with cross-attention and the proposed conditional encoding generally outperform other methods under noisy conditions, although the deterministic method excels with noiseless data. Additionally, both the diffusion models and the deterministic method surpass the numerical approach in accuracy and computational cost for the steady problem. We also demonstrate the ability of the model to capture possible reconstructions and improve the accuracy of fused results in covariance-based correction tasks using ensemble sampling.

   @article{ZHUANG2025117623,
     title = {Spatially-aware diffusion models with cross-attention for global field reconstruction with sparse observations},
     volume = {435},
     issn = {0045-7825},
     url = {https://www.sciencedirect.com/science/article/pii/S0045782524008776},
     doi = {https://doi.org/10.1016/j.cma.2024.117623},
     journal = {Computer Methods in Applied Mechanics and Engineering},
     author = {Zhuang, Yilin and Cheng, Sibo and Duraisamy, Karthik},
     year = {2025},
     keywords = {Diffusion model, Generative AI, Global field reconstruction, Inverse problems},
     pages = {117623},
   }

3. SISC

   

   CoCoGen: Physically Consistent and Conditioned Score-Based Generative Models for Forward and Inverse Problems

   Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy

   SIAM Journal on Scientific Computing, 2025

   Abs DOI arXiv Bib Code

   

   Recent advances in generative artificial intelligence have had a significant impact on diverse domains spanning computer vision, natural language processing, and drug discovery. This work extends the reach of generative models into physical problem domains, particularly addressing the efficient enforcement of physical laws and conditioning for forward and inverse problems involving partial differential equations (PDEs). Our work introduces two key contributions: firstly, we present an efficient approach to promote consistency with the underlying PDE. By incorporating discretized information into score-based generative models, our method generates samples closely aligned with the true data distribution, showcasing residuals comparable to data generated through conventional PDE solvers, significantly enhancing fidelity. Secondly, we showcase the potential and versatility of score-based generative models in various physics tasks, specifically highlighting surrogate modeling as well as probabilistic field reconstruction and inversion from sparse measurements. A robust foundation is laid by designing unconditional score-based generative models that utilize reversible probability flow ordinary differential equations. Leveraging conditional models that require minimal training, we illustrate their flexibility when combined with a frozen unconditional model. These conditional models generate PDE solutions by incorporating parameters, macroscopic quantities, or partial field measurements as guidance. The results illustrate the inherent flexibility of score-based generative models and explore the synergy between unconditional score-based generative models and the present physically-consistent sampling approach, emphasizing the power and flexibility in solving for and inverting physical fields governed by differential equations, and in other scientific machine learning tasks.

   @article{jacobsenCoCoGenPhysicallyConsistentConditioned2024,
     title = {{CoCoGen}: {Physically} {Consistent} and {Conditioned} {Score}-{Based} {Generative} {Models} for {Forward} and {Inverse} {Problems}},
     shorttitle = {{CoCoGen}},
     author = {Jacobsen, Christian and Zhuang, Yilin and Duraisamy, Karthik},
     journal = {SIAM Journal on Scientific Computing},
     volume = {47},
     number = {2},
     pages = {C399--C425},
     year = {2025},
     doi = {10.1137/24M1636071},
     url = {https://epubs.siam.org/doi/10.1137/24M1636071},
     keywords = {Computer Science - Machine Learning},
   }

2024

1. Int. J. Multiph. Flow

   Machine learning and physics-driven modelling and simulation of multiphase systems

   Nausheen Basha, Rossella Arcucci, Panagiota Angeli, and 31 more authors

   International Journal of Multiphase Flow, 2024

   Abs DOI Bib

   

   We highlight the work of a multi-university collaborative programme, PREMIERE (PREdictive Modelling with QuantIfication of UncERtainty for MultiphasE Systems), which is at the intersection of multi-physics and machine learning, aiming to enhance predictive capabilities in complex multiphase flow systems across diverse length and time scales. Our contributions encompass a variety of approaches, including the Design of Experiments for nanoparticle synthesis optimisation, Generalised Latent Assimilation models for drop coalescence prediction, Bayesian regularised artificial neural networks, eXtreme Gradient Boosting for microdroplet formation prediction, and a sub-sampling based adversarial neural network for predicting slug flow behaviour in two-phase pipe flows. Additionally, we introduce a generalised latent assimilation technique, Long Short-Term Memory networks for sequence forecasting mixing performance in stirred and static mixers, active learning via Bayesian optimisation to recover coalescence model parameters for high current density electrolysers, Gaussian process regression for drop size distribution predictions for sprays, and acoustic emission signal inversion using gradient boosting machines to characterise particle size distribution in fluidised beds. We also offer perspectives on the development of a shape optimisation framework that leverages the use of a multi-fidelity multiphase emulator. The results presented have applications in chemical synthesis, microfluidics, product manufacturing, and green hydrogen generation.

   @article{BASHA2024104936,
     title = {Machine learning and physics-driven modelling and simulation of multiphase systems},
     volume = {179},
     issn = {0301-9322},
     url = {https://www.sciencedirect.com/science/article/pii/S0301932224002131},
     doi = {https://doi.org/10.1016/j.ijmultiphaseflow.2024.104936},
     journal = {International Journal of Multiphase Flow},
     author = {Basha, Nausheen and Arcucci, Rossella and Angeli, Panagiota and Anastasiou, Charitos and Abadie, Thomas and Casas, César Quilodrán and Chen, Jianhua and Cheng, Sibo and Chagot, Loïc and Galvanin, Federico and Heaney, Claire E. and Hossein, Fria and Hu, Jinwei and Kovalchuk, Nina and Kalli, Maria and Kahouadji, Lyes and Kerhouant, Morgan and Lavino, Alessio and Liang, Fuyue and Nathanael, Konstantia and Magri, Luca and Lettieri, Paola and Materazzi, Massimiliano and Erigo, Matteo and Pico, Paula and Pain, Christopher C. and Shams, Mosayeb and Simmons, Mark and Traverso, Tullio and Valdes, Juan Pablo and Wolffs, Zef and Zhu, Kewei and Zhuang, Yilin and Matar, Omar K},
     year = {2024},
     keywords = {Hybrid methods, Machine Learning, Microfluidics, Multi-fidelity, Multiphase, Numerical simulations},
     pages = {104936},
   }

2. CMAME

   

   Multi-domain encoder–decoder neural networks for latent data assimilation in dynamical systems

   Sibo Cheng, Yilin Zhuang, Lyes Kahouadji, and 4 more authors

   Computer Methods in Applied Mechanics and Engineering, 2024

   Abs DOI Bib

   

   High-dimensional dynamical systems often require computationally intensive physics-based simulations, making full physical space data assimilation impractical. Latent data assimilation methods perform assimilation in reduced-order latent space for efficiency but struggle with complex, nonlinear state-observation mappings. Recent solutions like Generalized Latent Data Assimilation (GLA) and Latent Space Data Assimilation (LSDA) address heterogeneous latent spaces by incorporating surrogate mapping functions but introduce computational costs and uncertainties. Furthermore, current algorithms that integrate data assimilation and deep learning still face limitations when it comes to handling non-explicit mapping functions. To address these challenges, this paper introduces a novel deep-learning-based data assimilation scheme, named Multi-domain Encoder–Decoder Latent Data Assimilation (MEDLA), capable of handling diverse data sources by sharing a common latent space. The proposed approach significantly reduces the computational burden since the complex mapping functions are mimicked by the multi-domain encoder–decoder neural network. It also enhances assimilation accuracy by minimizing interpolation and approximation errors. Extensive numerical experiments from three different test cases assess MEDLA’s performance in high dimensional dynamical systems, benchmarking it against state-of-the-art latent data assimilation methods. The numerical results consistently underscore MEDLA’s superiority in managing multi-scale observational data and tackling intricate, non-explicit mapping functions.

   @article{CHENG2024117201,
     title = {Multi-domain encoder–decoder neural networks for latent data assimilation in dynamical systems},
     volume = {430},
     issn = {0045-7825},
     url = {https://www.sciencedirect.com/science/article/pii/S0045782524004572},
     doi = {https://doi.org/10.1016/j.cma.2024.117201},
     journal = {Computer Methods in Applied Mechanics and Engineering},
     author = {Cheng, Sibo and Zhuang, Yilin and Kahouadji, Lyes and Liu, Che and Chen, Jianhua and Matar, Omar K. and Arcucci, Rossella},
     year = {2024},
     keywords = {Data assimilation, Data fusion, Deep learning, Dynamical systems},
     pages = {117201},
   }

2023

1. INDIN 2023

   

   Semi-supervised Variational Autoencoders for Regression: Application to Soft Sensors

   Yilin Zhuang^*, Zhuobin Zhou^*, Burak Alakent, and 1 more author

   In 2023 IEEE 21st International Conference on Industrial Informatics (INDIN), Jul 2023

   ISSN: 2378-363X

   Abs DOI arXiv Bib Code

   

   We present the development of a semi-supervised regression method using variational autoencoders (VAE) for soft sensing of process quality variables. Recently, use of VAEs was proposed for regression applications based on variational inference. In this work, We extend this approach of supervised VAEs for regression to make it learn from both labelled and unlabelled data leading to a semi-supervised VAE for regression (SSVAER) formulation. The probabilistic regressor resulting from the variational approach makes it possible to estimate the variance of the predictions simultaneously, which provides a means for online uncertainty quantification for soft sensors. We provide an extensive comparative study of SSVAER with previously proposed semi-supervised learning methods on two soft sensing benchmark problems using fixed-size datasets, where we vary the percentage of labelled data available for training. In these experiments, SSVAER achieves the lowest test errors in 11 of the 20 studied cases, compared to other methods where the second best method gets 4 lowest test errors out of the 20.

   @inproceedings{zhuangSemisupervisedVariationalAutoencoders2023,
     title = {Semi-supervised {Variational} {Autoencoders} for {Regression}: {Application} to {Soft} {Sensors}},
     shorttitle = {Semi-supervised {Variational} {Autoencoders} for {Regression}},
     url = {https://ieeexplore.ieee.org/document/10218227},
     doi = {10.1109/INDIN51400.2023.10218227},
     urldate = {2025-02-23},
     booktitle = {2023 {IEEE} 21st {International} {Conference} on {Industrial} {Informatics} ({INDIN})},
     author = {Zhuang, Yilin and Zhou, Zhuobin and Alakent, Burak and Mercangöz, Mehmet},
     month = jul,
     year = {2023},
     note = {ISSN: 2378-363X},
     keywords = {Estimation, Job shop scheduling, Neurons, Semi-supervised learning, Semisupervised learning, soft sensors, Soft sensors, Training, Uncertainty, uncertainty analysis, variational autoencoder},
     pages = {1--8},
   }

2022

1. Lab on a Chip

   

   Ensemble latent assimilation with deep learning surrogate model: application to drop interaction in a microfluidics device

   Yilin Zhuang, Sibo Cheng, Nina Kovalchuk, and 4 more authors

   Lab on a chip, 2022

   Publisher: The Royal Society of Chemistry

   Abs DOI Bib

   

   A major challenge in the field of microfluidics is to predict and control drop interactions. This work develops an image-based data-driven model to forecast drop dynamics based on experiments performed on a microfluidics device. Reduced-order modelling techniques are applied to compress the recorded images into low-dimensional spaces and alleviate the computational cost. Recurrent neural networks are then employed to build a surrogate model of drop interactions by learning the dynamics of compressed variables in the reduced-order space. The surrogate model is integrated with real-time observations using data assimilation. In this paper we developed an ensemble-based latent assimilation algorithm scheme which shows an improvement in terms of accuracy with respect to the previous approaches. This work demonstrates the possibility to create a reliable data-driven model enabling a high fidelity prediction of drop interactions in microfluidics device. The performance of the developed system is evaluated against experimental data (i.e., recorded videos), which are excluded from the training of the surrogate model.

   @article{D2LC00303A,
     title = {Ensemble latent assimilation with deep learning surrogate model: application to drop interaction in a microfluidics device},
     volume = {22},
     url = {http://dx.doi.org/10.1039/D2LC00303A},
     doi = {10.1039/D2LC00303A},
     number = {17},
     journal = {Lab on a chip},
     author = {Zhuang, Yilin and Cheng, Sibo and Kovalchuk, Nina and Simmons, Mark and Matar, Omar K. and Guo, Yi-Ke and Arcucci, Rossella},
     year = {2022},
     note = {Publisher: The Royal Society of Chemistry},
     pages = {3187--3202},
   }

2. Comput. Ind.

   

   A hybrid data-driven and mechanistic model soft sensor for estimating CO2 concentrations for a carbon capture pilot plant

   Yilin Zhuang, Yixuan Liu, Akhil Ahmed, and 4 more authors

   Computers in Industry, 2022

   Abs DOI Bib Code

   

   Integrating post-combustion carbon capture and storage (CCS) facilities into fossil fuel power plants is considered an important step for reaching global carbon emission reduction targets. When the number of gas analyzers in such CCS units is limited, variations in the load of the power plant pose a challenge to determine the gaseous CO2 concentration profile in the absorber. A dynamic hybrid model for estimating the carbon capture absorber’s gaseous CO2 concentration profile has been proposed in this study. The model is built using actual process data collected from a carbon capture pilot plant and it combines data-driven and reaction kinetic (mechanistic) modeling approaches to act as a soft sensor along the absorber column. A subset of the process data is used for training the data-driven models and for estimating the parameters of the mechanistic model respectively. Dimensionality reduction techniques are applied to the data-driven model to reduce the input size and hence the size of the dynamic model elements. The outputs of the two models are fused by comparing the computed covariance matrices. A particular challenge for this work is that the collected process data has missing spatial labels and temporal values for the CO2 concentration measurements. The presented models are obtained using an encoding and interpolation approach for the missing information. For comparison, an alternative approach based on semi-supervised learning has been implemented. The performance of the resulting soft sensors is verified by using process data from previously unseen operating conditions. The soft sensor based on the proposed hybrid model outperforms the soft sensor trained by semi-supervised autoencoder. Overall the results indicate that the proposed approach can estimate the CO2 concentration percentage with an average root mean squared error of 0.123.

   @article{ZHUANG2022103747,
     title = {A hybrid data-driven and mechanistic model soft sensor for estimating {CO2} concentrations for a carbon capture pilot plant},
     volume = {143},
     issn = {0166-3615},
     url = {https://www.sciencedirect.com/science/article/pii/S0166361522001440},
     doi = {https://doi.org/10.1016/j.compind.2022.103747},
     journal = {Computers in Industry},
     author = {Zhuang, Yilin and Liu, Yixuan and Ahmed, Akhil and Zhong, Zhengang and del Rio Chanona, Ehecatl A. and Hale, Colin P. and Mercangöz, Mehmet},
     year = {2022},
     keywords = {Carbon capture pilot plant, Data-driven modeling, Long short term memory, Soft sensor},
     pages = {103747},
   }

2020

1. APS DFD

   Benchmarking of CFD solvers for the simulation of two-phase jets

   Yilin Zhuang, Gabriel FN Gonçalves, Cristian Ricardo Constante Amores, and 2 more authors

   In APS division of fluid dynamics meeting abstracts, 2020

   Abs

   Multiphase flows have received significant interest due to their occurrence in a multitude of natural and industrial applications. In this study, we perform the benchmarking of solvers for the simulation of two-phase jets. The Navier-Stokes equation are solved under the assumption of the single-fluid formulation and therefore, the interfacial location must be also determined as part of the solution. We evaluate solvers based on both interface-capturing methods (such as Basilisk and OpenFOAM) and interface-tracking methods. We consider the scenario of a water drop formation from a cylindrical nozzle and the accuracy of the different solvers is compared with experimental results, in terms of jet-shape and drop-size. A thorough study in terms of the computational requirements (e.g., grid resolution) and the computational cost is also performed.