PhD Position F/M [Campagne DOC BMI-NF-GRA-2024] Joint 3D reconstruction and correspondence computation of deforming objects from multi-view 2D videos
Type de contrat : Fixed-term contract
Niveau de diplôme exigé : Graduate degree or equivalent
Fonction : PhD Position
A propos du centre ou de la direction fonctionnelle
The Centre Inria de l’Université de Grenoble groups together almost 600 people in 22 research teams and 7 research support departments.
Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (Université Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.
The Centre Inria de l’Université Grenoble Alpe is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.
Contexte et atouts du poste
The proposed research project will be carried out within the Morpheo team, led by Jean-Sebastien Franco, at Inria Grenoble Rhône-Alpes. The main research focus of this team is the capture and the analysis of shapes in motion. To this objective, Morpheo has access to the Kinovis platform, a unique multi-camera system with a large capture volume that allows to model dynamic scenes with high spatial and temporal resolutions.
This PhD project is part of Morpheo team's long-term research goal to develop tools for the analysis of 3D objects in motion. Such tools are of interest in various applications ranging from virtual telepresence to entertainment, e.g. motion synthesis for virtual changerooms or content generation. In this project, we are more particularly interested in jointly reconstructing the 3D geometry of a deforming object and finding corresponding points over time, based on information learned from example sequences.
The Inria Grenoble offers high-standard research facilities and is located within the very dynamic region of Grenoble. Grenoble is a city that hosts many international students and researchers. The city is located in the French Alps, and offers easy access to great outdoors activities such as hiking and skiing. Paris can be reached by train in three hours.
Mission confiée
Description of the research project:
Reconstructing 3D dynamically deforming objects from synchronized 2D video input is a well researched problem in computer vision and graphics with applications ranging from virtual telepresence, such as virtual change rooms, to entertainment, where realistic content generation is important. Existing reconstruction approaches allow for highly detailed geometric 3D reconstructions computed independently at each time instant, such as [1,2,3]. While these results are impressive, no information on the dynamic evolution of the 3D surface is encoded in the resulting reconstruction.
Another well-studied problem in computer vision is to compute meaningful correspondence information between deforming surfaces. This information is crucial when the goal is to analyze or modify a dynamically deforming shape coherently over time, for instance when changing its appearance. Existing works often operate on 3D surfaces after the reconstruction phase, such as [4,5].
The reconstruction and correspondence problems are interdependent. On the one hand, having faithful 3D reconstructions facilitates the task of correspondence methods, as robustness to different types of noise is challenging to obtain. On the other hand, when correspondence information is known, it can be used to improve 3D reconstructions, for instance by transferring surface parts occluded in the input videos of a frame from its previous time frame. Hence, there have been efforts to extend neural radiance fields [1], suitable for static reconstruction, to dynamically deforming surfaces, e.g. [6,7].
In this Ph.D. offer, we aim to study 3D reconstruction and correspondence jointly. The goal is to model the 3D surface as object that undergoes deformations that can be characterized by a deformation model, similar to the idea presented in [4], as this hypothesis helps to regularize the ill-posed problem. This allows for the 3D reconstruction to benefit from the correspondence information and vice versa.
[1] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng. Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV, 2020.
[2] Briac Toussaint, Maxime Genisson, Jean-Sébastien Franco. Fast Gradient Descent for Surface Capture Via Differentiable Rendering. International Conference on 3DV, 2022
[3] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, Jerome Revaud. DUSt3R: Geometric 3D Vision Made Easy. arXiv 2312.14132, 2023.
[4] Aymen Merrouche, João Regateiro, Stefanie Wuhrer, Edmond Boyer. Deformation-Guided Unsupervised Non-Rigid Shape Matching. BMVC, 2023.
[5] S Attaiki, M Ovsjanikov. NCP: Neural correspondence prior for effective unsupervised shape matching. NeurIPS, 2022.
[6] Albert Pumarola, Enric Corona, Gerard Pons-Moll, Francesc Moreno-Noguer. D-NeRF: Neural Radiance Fields for Dynamic Scenes. CVPR, 2021.
[7] Atsuhiro Noguchi, Xiao Sun, Stephen Lin, Tatsuya Harada. Neural Articulated Radiance Field. ICCV, 2021.
Principales activités
Required skills:
- Master degree in Computer Science or Applied Mathematics
- Creative and highly motivated
- Solid programming skills
- Solid mathematical background, especially in geometry, linear algebra, statistics
- Language requirements: fluent spoken and written English
- Prior knowledge in the areas of machine learning, computer vision, computer graphics or computational geometry is a plus
The project will be supervised by
- Jean-Sébastien Franco (jean.sebastien.franco@inria.fr) and
- Stefanie Wuhrer (stefanie.wuhrer@inria.fr)
Compétences
See above.
Avantages
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (90 days / year) and flexible organization of working hours (except for intership)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage under conditions
Rémunération
1st and 2nd year: 2 100 euros gross salary /month
3rd year: 2 190 euros gross salary / month
Informations générales
- Thème/Domaine :
Vision, perception and multimedia interpretation
Statistics (Big data) (BAP E) - Ville : Montbonnot
- Centre Inria : Centre Inria de l'Université Grenoble Alpes
- Date de prise de fonction souhaitée : 2024-10-01
- Durée de contrat : 3 years
- Date limite pour postuler : 2024-04-30
Attention: Les candidatures doivent être déposées en ligne sur le site Inria. Le traitement des candidatures adressées par d'autres canaux n'est pas garanti.
Consignes pour postuler
CV, cover letter, Master's grades, a letter of recommendation from the Master's course supervisor (or equivalent), possibly a letter of recommendation from the master's supervisor.
Sécurité défense :
Ce poste est susceptible d’être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n°2011-1425 relatif à la protection du potentiel scientifique et technique de la nation (PPST). L’autorisation d’accès à une zone est délivrée par le chef d’établissement, après avis ministériel favorable, tel que défini dans l’arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l’annulation du recrutement.
Politique de recrutement :
Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap.
Contacts
- Équipe Inria : MORPHEO
-
Directeur de thèse :
Wuhrer Stefanie / stefanie.wuhrer@inria.fr
L'essentiel pour réussir
See above.
A propos d'Inria
Inria est l’institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l’interface d’autres disciplines. L’institut fait appel à de nombreux talents dans plus d’une quarantaine de métiers différents. 900 personnels d’appui à la recherche et à l’innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'efforce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.