PhD Position F/M Prediction of human whole-body motion for anticipation in human-robot interaction using multimodal, contextual models and generative AI
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
Context
The position is funded by the PEPR O2R, a national French program to advance research in robotics which reunites several French laboratories in robotics, AI, and Social and Human Sciences.
In this context, the LARSEN team in INRIA is collaborating with CEA, to advance the anticipation skills of robots interacting with humans. One goal is to estimate the human intention of motion using the sensors onboard the robot (cameras, IMUs, microphones, etc), considering the contextual information extracted from vision or audio processing.
The successful candidate will join our dynamic research team and contribute to cutting-edge advancements in robotics and will be jointly supervised by INRIA (Serena Ivaldi and Jean-Baptiste Mouret) and CEA (Bertrand Luvison).
The PhD thesis will be physically based in Nancy, but the student will spend a short visiting period in CEA too.
The PhD candidate will participate in the exchange activities of PEPR, including hackathons, national conferences, and summer schools.
The student will also have the opportunity to perform the research using laboratory robotic assets including the omniTiago++ robot, the humanoid bipedal robot TALOS.
Assignment
The objective of this Ph.D. is to propose new algorithms to accurately predict the future whole-body motion of a human, which is critical for human-robot collaboration. This has a direct application in social robots co-existing or interacting with humans in their environment, in exoskeletons, and in the direct teleoperation of robots with different degrees of anthropomorphism.
The thesis will target several advances in human motion prediction, such as predicting the intention of motion at different time scales, with different families of models, adding physical priors to guarantee the prediction follows the model of the human and physics constraints (e.g., the human is balanced, with the feet on the floor), and incorporating external sources of information to improve the prediction while taking the environment into account (objects nearby, payloads, obstacles from RGB-D sensors, etc.).
The level of physical interaction in this thesis will be minimal, as the focus will be particularly on situations when the onset of action must be detected to ensure a prompt anticipatory action from the robot. As an example, handovers or collaborative lifting/carrying will be studied.
The software and methods produced by Ph.D. will be a basic “building block" for many interaction scenarios addressed in the PEPR project: developing open software that can be re-used in other projects will be of paramount importance.
Main activities
- Write code to perform experiments with robots
- Write software documentation
- Write an annual activity report
- Conduct experiments in the real environment, with real robots: Tiago++
- Write scientific articles and reports
- Participate to national and European projects events (meetings, hackathons) when appropriate
Skills
- Experience with machine learning, vision, or real robots is necessary.
- Experience with LLMs, visual models, generative AI, diffusion models is a plus.
- Familiarity with robotic platforms and simulation environments (e.g., ROS2, Gazebo, MuJoCo, CasADi, Eigen, Pinocchio, Nvidia Isaac GYM) is a plus.
- Proficiency in programming languages such as C/C++ and/or Python is required.
- Ability to work independently and as part of a team.
- Good communication and writing skills in English.
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
Remuneration
2100€ gross/month the 1st year
General Information
- Theme/Domain :
Robotics and Smart environments
Software engineering (BAP E) - Town/city : Villers lès Nancy
- Inria Center : Centre Inria de l'Université de Lorraine
- Starting date : 2024-09-02
- Duration of contract : 3 years
- Deadline to apply : 2024-05-12
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.
Instruction to apply
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Contacts
- Inria Team : LARSEN
-
PhD Supervisor :
Ivaldi Serena / serena.ivaldi@inria.fr
The keys to success
Master's degree (or equivalent) in Robotics, AI, Computer Science, or a related field.
About Inria
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.