Logo Inria

PhD Position F/M Learning bimanual robot skills from human demonstrations and natural language

Contract type : Fixed-term contract

Level of qualifications required : Graduate degree or equivalent

Fonction : PhD Position

Context

The team LARSEN is involved in the European project euROBIN. One of the main goals of the project is to advance cognition-enabled transferable embodied AI. Scientifically, it will substantially advance four core scientific topics: InterAct, Learning transfer, Transferable knowledge, and Human-center transfer. Three robotics domains are investigated: manufacturing, outdoor and personal robotics. In this project, Inria is leading the personal robotics challenge.

In this project, INRIA is leading the personal Robotics Challenge, where bimanual manipulators and humanoid robots must execute a variety of complex manipulation, navigation and interaction tasks in a household scenario. Some of these tasks involve unloading a dishwasher, opening a fridge to take an object, folding clothes, carrying and handing objects to humans.

Annual hackathons are organized to foster collaborations among European teams on robotics challenges.

Assignment

In this context, we seek a PhD candidate to develop algorithms for learning complex whole-body skills for robots by combining human demonstrations and natural language instructions. Human demonstrations will be preferably retrieved via direct teleoperation or retargeting from human motions. In the team, both solutions have been used. We would like to explore natural language instructions to provide online corrections, re-planning and control setting modification, ultimately to generate situated, context-aware trajectories from speech. 

The candidate will be working in an international team with engineers, postdocs and permanent researchers, in a dynamic environment with plenty of robots: Tiago++, Talos, iCub, exoskeletons, mini-drones among others, including top robotics equipment and facilities.

The PhD candidate will mostly work with Serena Ivaldi and Jean-Baptiste Mouret. 

References: 

[1] Penco L, Clément B, Modugno V, Hoffman EM, Nava G, Pucci D, Tsagarakis NG, Mouret JB, Ivaldi S. Robust real-time whole-body motion retargeting from human to humanoid. In 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) 2018 Nov 6 (pp. 425-432). IEEE. 

[2] Penco, L.; Mingo Hoffman, E.; Modugno, V.; Gomes, W.; Mouret, J.-B.; Ivaldi, S. (2020) Learning Robust Task Priorities and Gains for Control of Redundant Robots. IEEE Robotics and Automation Letters.

[3] Penco, L.; Scianca, N.; Modugno, V.; Lanari, L.; Oriolo, G.; Ivaldi, S. (2019) A Multi-Mode Teleoperation Framework for Humanoid Loco-Manipulation. IEEE Robotics & Automation Magazine. Volume 26, Issue 4, Pages 73-82. doi: 10.1109/MRA.2019.2941245

Main activities

  • Write code to perform experiments with robot 
  • Write software documentation
  • Conduct experiments in the real environment, with real robots: Tiago++
  • Write scientific articles and reports
  • Participate to national and European projects events (meetings, hackathons) when appropriate

Skills

Candidates should be expert in at least one of these areas:

  • Robotics (kinematics, dynamics)
  • Whole-body Control
  • Machine learning
  • Computer vision
  • Modern C++ programming 
  • Python
  • Natural language models / training 

Excellent communication skills and English are required. Knowledge of the French language is not a requirement.

The ideal candidate is a team player, with good communication skills.

Other valued appreciated : proactivity, autonomy, curiosity.

Benefits package

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage

Remuneration

2082 gross/month for the 1st and 2nd years. 2190€ gross/month for the 3rd year.