2022-04697 - PhD Position F/M Data-driven anticipation for whole-body teleoperation of humanoids

Level of qualifications required : Graduate degree or equivalent

Fonction : PhD Position

Context

Our general objective is to accurately teleoperate humanoid robots from a whole-body motion capture system. Like the human operator, the robot must carefully synchronize its many degrees of freedom to achieve the desired motion while keeping its balance (whole body control). 

We recently obtained promising results with the iCub humanoid robot for whole-body teleoperation [1,2,4]. This technique (called “motion retargeting”) does not anticipate the motion of the operator, as it aims at matching the positions of the end-effectors at each time step. For instance, the robot does not know when the operator will stop its motion. This lack of anticipation can, however, cause substantial tracking errors because the kinematics and the dynamics of the robot do not match those of the human perfectly: for instance, the robot might need more anticipation that the human to stop the trajectory of its hands because it is heavier. In addition, the absence of anticipation forces the robot to always be in static balance, since the motion can stop at any instant. 

 

Assignment

The objective of this PhD is to design a whole-body teleoperation system that anticipates the motion of the operator. It will be demonstrated on the TALOS humanoid robot of Inria Nancy - Grand Est / LORIA, a full-sized, state-of-the-art humanoid robot. The PhD will combine two highly dynamic scientific fields:

  • machine learning to learn trajectories of the humans and build predictors of future motions given current motion;
  • whole-body model predictive control [3] to use the predictions to improve the accuracy and the balance of the robot, especially when performing fast movements.

The first challenge will be to design accurate motion prediction algorithms based on the state-of-the-art in machine learning (LSTM, recurrent neural networks, etc.). It will then be necessary to extend them to take the context into account (obstacles, potential targets, etc.).

The applicant must have strong knowledge of robotics and machine learning, as well as strong C++ skills (the robot is programmed in C++ and real-time performance is required).

This PhD will be jointly supervised by Jean-Baptiste Mouret and Serena Ivaldi. 

References:

[1] Penco L, Clément B, Modugno V, Hoffman EM, Nava G, Pucci D, Tsagarakis NG, Mouret JB, Ivaldi S. Robust real-time whole-body motion retargeting from human to humanoid. In2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) 2018 Nov 6 (pp. 425-432). IEEE.

[2] Penco L, Clément B, Modugno V, Hoffman EM, Nava G, Pucci D, Tsagarakis NG, Mouret JB, Ivaldi S. Robust real-time whole-body motion retargeting from human to humanoid. In2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) 2018 Nov 6 (pp. 425-432). IEEE.

[3] Koenemann J, Del Prete A, Tassa Y, Todorov E, Stasse O, Bennewitz M, Mansard N. Whole-body model-predictive control applied to the HRP-2 humanoid. In2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015 Sep 28 (pp. 3346-3351). IEEE.

 [4] Penco, L., Mouret, J. B., & Ivaldi, S. (2021). Prescient teleoperation of humanoid robots. arXiv preprint arXiv:2107.01281.

Main activities

Main activities:

  • Write code to perform experiments with robot
  • Perform experiments in simulation
  • Perform experiments with the Talos robot
  • Write scientific articles

Skills

  • Robotics (kinematics, dynamics)
  • Control (in particular, model predictive control)
  • Machine learning
  • Modern C++ programming

Benefits package

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage

Remuneration

1982€ gross/month for 1st and 2nd year. 2085€ gross/month for 3rd year.

Monthly salary after taxes : around 1596,05€ for 1st and 2nd year. 1678,99€ for 3rd year. (medical insurance included).