Research Engineer: Detailed riggable humans from multi-view video M/F

The Inria Grenoble research center groups together almost 600 people in 27 research teams and 8 research support departments.

Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (University Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.

Inria Grenoble is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.

Within the framework of a partnership 

• The project's findings will contribute to a follow up 4-year BPI transfer project (Banque Publique d'investissement) with a consortium of French production companies. The project is liable to begin during this contract period, with discussions with involved partners

Context

Many works nowadays provide solutions for the avatarization process, i.e. obtaining a 3D, animatable model from one or several images. This is a very hard problem as shape fidelity, animatability, fast computation time and low number of required input cameras are all desirable, but hardly realizable simultaneously. For example, obtaining plausible models from a single camera video is nowadays feasable, but often at the cost of shape quality due to the use of low-parametric models such as SMPL. Using many videos for redundancy can allow to acquire more detail, but at the expense of computation speed. And all this detail needs to be animatable, which gets more complex with the scale of detail (i.e. millimeter shape with only a human kinematic rig), again putting a burden on the model and its computation time.

Mission

In recent years the Morpheo team has come up with very precise multi-view reconstruction approaches [1].
In this project, we wish to examine the problem of animating this type of detailed model and rig it, by exploring the stream of recent methods.

During his activity, the candidate is expected to tackle the following tasks

• establish a more complete bibliography of relevant methods based on the initial suggested references
• propose and discuss likely and realizable methodological and architecture innovations / reparametrizations that allow to rig and estimate a detailed animated model from images, grounded in this existing work
• propose and discuss dataset enhancements that would enrich the training toward better performance for these tasks
• demonstrate a proof of concept implementation of an envisioned solution discussed with team research advisers
• identify existing datasets that are relevant for comparative evaluation of performance of his proposals. In-house datasets such as 4DHumanOutfit[2]

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking (90 days / year) and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Complementary health insurance under conditions

From 2,692 € (depending on experience and qualifications).