PhD Position F/M No-brain-shift and Comprehensive Neurosurgical Navigation using computer vision

The Inria centre at Université Côte d'Azur includes 37 research teams and 8 support services. The centre's staff (about 500 people) is made up of scientists of different nationalities, engineers, technicians and administrative staff. The teams are mainly located on the university campuses of Sophia Antipolis and Nice as well as Montpellier, in close collaboration with research and higher education laboratories and establishments (Université Côte d'Azur, CNRS, INRAE, INSERM ...), but also with the regiona economic players.

With a presence in the fields of computational neuroscience and biology, data science and modeling, software engineering and certification, as well as collaborative robotics, the Inria Centre at Université Côte d'Azur  is a major player in terms of scientific excellence through its results and collaborations at both European and international levels.

The candidate will be part of the CAMIN team at INRIA.

The PhD thesis will be funded conjointly by INRIA and INSERM.

The aim of this project is (i) to reconstruct precisely, automatically and conti-
nuously the volume of the brain exposed by craniotomy during neurosurgery and (ii)
to reposition this volume distorted by the drop in intracranial pressure and resection
in standard imaging (MRI, PET-Scan).The ambition is to propose a corrected
and complete neuronavigation that is better suited to research.

Brain surgery is essentially based on planning the procedure on the basis of pre-operative anatomical images (MRI, PET-Scan). Intra-operatively, apart from surgery on awake patients, there is no relevant anatomical imaging to guide surgery online, mainly because of (i) the collapse of the brain with the drop in intracranial pressure that occurs when the dura mater is opened and (ii) the substantial resection (sometimes more than a hundred $cm^3$ for tumour resection). This overall phenomenon, known as "brain shift", leads to shifts of the order of a centimetre between the actual position of a surgical tool and its position in the anatomical images. Current neuro-navigation systems, based on classic movement analysis tools (infra-red localisation of passive markers):

• cannot compensate for the "brain shift" . However, when the surgical intervention reaches the white matter fascicles, sub-millimetre precision would be necessary to try to limit the resection of fascicles that are critical for cerebral function.
• What's more, these solutions do not allow continuous monitoring of anatomical changes and the various surgical procedures. No access to historical data is currently possible. This makes it impossible to reconstruct resection cavities, stimulation points and tool trajectories.
• Furthermore, when intra-operative neural monitoring is necessary to identify structural connectivity by electrophysiological measurements and requires the placement of electrodes, the precise location of these electrodes and the associated electrical stimulation points on standard imaging is not possible and remains reconstructed a posteriori with considerable imprecision, which limits the interpretation and clinical relevance of the data collected. \cite{rossel23}.

The ultimate aim of this project is to resolve these 3 limitations by changing the technological paradigm. In concrete terms, based on two calibrated video streams (stereoscopic vision probably embarked by the neurosurgeon), the approach will consist of extracting several pieces of information in the space of the camera images: fixed points of interest in the scene, mobile anatomical landmarks linked to the brain shift, 6D pose of the surgical tools. This feature extraction stage will be carried out using various machine learning tools (i.e. convolutional neural networks, transformers). The 2D information will then be combined by triangulation to obtain a 3D+time description of the surgery in a fixed anatomical frame of reference (linked to the cranial cavity).

This information will be used to estimate mobile quantities:

• the 3D volume representing the surface of the bare brain and the resection cavity,
• the main blood vessel segments, which follow the main cortical sulci,
• electrocorticographic collection electrodes,
• 6D time trajectories of surgical tools

and quantitites \textbf{fixed} in the laboratory frame of reference (the patient's skull is fixed by a headrest).
The 3D representation of this data combined with the knowledge of fixed landmarks will enable it to be matched with T1 MRI images. This represents three key innovations compared with existing neuronavigation: A) the brain shift is corrected in the navigation data reprojected into the MRI, B) the trajectories of the surgical tools, the dynamics of the brainshift, etc. are available postoperatively, and C) the measurement and stimulation electrodes are precisely located throughout the surgery.

Automatic segmentation
The cerebral surface will be reconstructed i) by extracting the 2D coordinates of various anatomical and artificial points of interest (moved by the 'brain shit': blood vessels, surgical tags, stimulation electrodes and ii) by estimating the 6D pose of the various rigid tools that the surgeon handles \cite{moullet23} and \textbf{puts in contact with the brain} (suction, ultrasound fragmenter (CUSA), stimulation probe, various forceps). The operating principle is that, knowing the geometry of these tools a priori, we can use the partial vision provided by the stereoscopic vision device worn by the surgeon to deduce the hidden part (in the cerebral matter). By analysing the positional and velocity trajectories of the tips of these tools, we can precisely deduce the obstacles, i.e. the cerebral surface.

Reprojection of the brain surface in MRI
The aim is to match the surface obtained by computer vision and the various landmarks with the MRI data. The challenge is to model the geometry of the brain-shift in order to compensate for it mathematically and merge the 2 types of data.

Flow-Chart
These segmentations and reconstructions will be used continuously to (i) estimate the 'brain shift' and enrich the models that describe it, and (ii) move the surgical tools and electrodes (stimulation, measurement).
The volume of the resection cavity will be updated each time a surgical tool is placed in contact with the tissue.

We will consider two situations to compare their accuracy. A situation with "calibration" (movement by the neurosurgeon of a rigid tool on the surface of the brain) and without "calibration" (no particular action on the part of the neurosurgeon).
The development will be carried out in close collaboration with the neurosurgeon E. Mandonnet, to evaluate the practical and ergonomic aspects of the proposed solution.

Technological development
This project involves the development of an ergonomic wearable device fixed to the neurosurgeon's head for positioning a pair of stereoscopic RGB cameras. Initially, we will consider the portable case with a fixed distance between the two cameras for continuous stereoscopic vision of the craniotomy zone and minimising obturation of the cameras' field of view (optimal situation for image capture). If this situation proves unviable in the operating theatre or sub-optimal for the neurosurgeon, we will consider relocating the positioning of the cameras while maintaining their fixed distance and orientation, even if they are attached to moving objects in the operating theatre. For the time being, the solutions we are considering involve either attaching the two cameras to the arms of glasses or to a headband.

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

Duration: 36 months
Location: Sophia Antipolis, France
Gross Salary per month: 2100€ brut per month (year 1 & 2) and 2190€ brut per month (year 3)