###Multi-modal Synthetic Data Simulation for Image Guided Surgery
Project ID: 2228bd1190 (You will need this ID for your application)
Research Theme: Healthcare Technologies
UCL Lead department: Medical Physics and Biomedical Engineering
Lead Supervisor: Thomas Dowrick
Project Summary:
A majority of researchers in Image Guided Surgery (IGS) are involved with machine learning in some form. However, the limited availability of clinical data for model training is a major blocker for development. Additionally, the time-consuming process of manually labelling data is especially challenging for medical data, as labelling typically requires the intervention of a trained clinician. In certain cases, such as stereo reconstruction or optical flow, ground truth data cannot be obtained at all.
The wider computer vision community has benefited from large open sourced datasets, including both real ( ImageNet, KITTI) and simulated (Scene Flow, Virtual KITTI) data. Synthetic data has the additional advantage of being able to generate more complex labels, without the time/cost overheads of manual labelling.
This project will build upon existing work, using Unity1 and Blender, which can produce large scale (100k+) photorealistic simulations of video and image data for IGS applications, to include multiple simulation modalities (CT/MRI/Ultrasound) within a single simulation framework. This will be achieved using a combination of ‘standard’ simulation tools (e.g. Sofa2), procedural texture generation and modelling (Blender Geometry Nodes), custom simulation software written during the project, and state of the art deep learning methods for data synthesis (Generative Adversarial Networks (GANs), Neural Radiance Fields (NERFs) and Diffusion Models). Applications which require more than one imaging modality, such as registration between ultrasound, CT or video data, are of particular interest for simulation, and provide an additional challenge, as data must be consistent across different modalities.
The project will provide scope for a) significant advancements in machine learning applications for IGS; b) development of novel software for surgical simulation, which can be deployed locally, as well as on cloud and HPC services; and c) the production of synthetic datasets that can be made available to the wider IGS community.