PhD Position F/M Geometrically-aware Vision Foundation Models

The Inria Saclay-Île-de-France Research Centre was established in 2008. It has developed as part of the Saclay site in partnership with Paris-Saclay University and with the Institut Polytechnique de Paris .

The centre has 40 project teams , 27 of which operate jointly with Paris-Saclay University and the Institut Polytechnique de Paris; Its activities occupy over 600 people, scientists and research and innovation support staff, including 44 different nationalities.

Within the framework of a partnership (you can choose between)

• not applicable,
• collaboration between 2 Inria teams: *****,
• collaboration ({team_Inria} and the start-up *****,
• project/programme/European fund *****,
• public with {French National Research Agency (ANR), local and regional authorities, academic partners, *****]
• value-creation and technology transfer contracts with *****

a package/model/prototype/application/interface/infrastructure/other specify *****
more specifically dedicated to *****.

Is regular travel foreseen for this post ? “Do not hesitate to make this known and to ensure that "travel expenses are covered within the limits of the scale in force".

The recent advent of large deep learning methods has brought tremendous progress in a broad range of Computer Vision tasks, such as image representation and generation. Robustness and generalisation, however, remain important challenges, as these models still lag behind human visual perception and understanding. Notably, humans excel at adapting to new environments and challenging perceptual conditions. Meanwhile, despite being trained on vast amounts of data, vision models often struggle with new objects or scenes. This limits their adaptation to real-world applications. A critical difference lies in the fact that humans benefit from evolving in and interacting with a 3D environment endowed with a rich structure that images lack. In particular, shapes are known to play a key role in human perception. 

This project aims to bridge the gap between human and machine perception by leveraging geometric structure and incorporating it into vision models. We will explore ways in which such information can be combined with extensive data-driven priors extracted from large image databases. One possibility is to encapsulate geometric information through inductive biases in the model architectures or, alternatively, by promoting geometric awareness through post-training fine-tuning methods. Finding computationally efficient, scalable, and robust methods to transfer geometric structure from shapes to models that operate on images will be an important goal of this thesis. Applications of geometrically-aware vision models are numerous, including asset generation, structured world modelling, and 3D scene reconstruction and understanding from images. This project will also explore applications and study the role of geometric priors in these contexts. 

• Develop novel approaches
• Prepare publications in top-venues (CVPR, ICCV, NeurIPS, SIGGRAPH, etc.)
• Present your work at conferences
• Prepare a dissertation

Technical skills and level required :

Languages :

Relational skills :

Other valued appreciated :

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

Monthly gross salary : 2.200 euros