###Model-based Optimization and Reinforcement Learning with Applications to Nuclear Fusion Research
Project ID: 2228bd1132 (You will need this ID for your application)
Research Theme: Artificial Intelligence and Robotics
UCL Lead department: Electronic and Electrical Engineering (EEE)
Lead Supervisor: Ilija Bogunovic
Project Summary:
Efficient learning from experimental data or simulations is of high interest in many real-world applications (in both science and engineering) where the goal is to identify optimal design (or policy) from a minimal number of costly trials. Consequently, Bayesian optimization (BO) and model-based reinforcement learning (RL) have received significant attention as a rapidly expanding class of powerful machine learning techniques that make use of surrogate models to learn and recommend new promising experiments to run (i.e., new data to collect). These methods are of particular interest in fusion research since they can simultaneously model and provide fast and accurate predictions of future nuclear plasma states and enable the planning and control of machine operations. Fusion promises to be an environmentally friendly part of the world’s future energy supply and has often been described as the ultimate energy source, hence, this is a perfect task for efficient and robust model-based RL and BO algorithms as these would allow effectively solving problems that would have previously taken months or years. Ultimately, this will save time and money, but will also significantly improve robustness when transferring from simulations to reality, or when designing plasma scenarios in newly-designed machines. The project will develop rigorous theory and algorithms for robust and efficient model-based RL and optimization. We will address challenges such as the sim-to-real gap, corrupted observations, safe exploration of parameter spaces, and distributional data shifts (e.g., when transferring to newly-designed machines). To acquire expert domain knowledge, we will actively collaborate with UKAEA and their data and research scientists. This project focuses on methodology and developing state-of-the-art algorithms. Therefore, we are looking for a student with strong machine learning, mathematical and practical background. Prior experience with PyTorch or JAX is required, please include a sample of a repository you have worked on in your application.