PhD Position F/M Learning bimanual robot skills from human demonstrations and natural language
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
Context
The team LARSEN is involved in the European project euROBIN. One of the main goals of the project is to advance cognition-enabled transferable embodied AI. Scientifically, it will substantially advance four core scientific topics: InterAct, Learning transfer, Transferable knowledge, and Human-center transfer. Three robotics domains are investigated: manufacturing, outdoor and personal robotics. In this project, Inria is leading the personal robotics challenge.
In this project, INRIA is leading the personal Robotics Challenge, where bimanual manipulators and humanoid robots must execute a variety of complex manipulation, navigation and interaction tasks in a household scenario. Some of these tasks involve unloading a dishwasher, opening a fridge to take an object, folding clothes, carrying and handing objects to humans.
Annual hackathons are organized to foster collaborations among European teams on robotics challenges.
Assignment
In this context, we seek a PhD candidate to develop algorithms for learning complex whole-body skills for robots by combining human demonstrations and natural language instructions. Human demonstrations will be preferably retrieved via direct teleoperation or retargeting from human motions. In the team, both solutions have been used. We would like to explore natural language instructions to provide online corrections, re-planning and control setting modification, ultimately to generate situated, context-aware trajectories from speech.
The candidate will be working in an international team with engineers, postdocs and permanent researchers, in a dynamic environment with plenty of robots: Tiago++, Talos, iCub, exoskeletons, mini-drones among others, including top robotics equipment and facilities.
The PhD candidate will mostly work with Serena Ivaldi and Jean-Baptiste Mouret.
References:
[1] Penco L, Clément B, Modugno V, Hoffman EM, Nava G, Pucci D, Tsagarakis NG, Mouret JB, Ivaldi S. Robust real-time whole-body motion retargeting from human to humanoid. In 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) 2018 Nov 6 (pp. 425-432). IEEE.
[2] Penco, L.; Mingo Hoffman, E.; Modugno, V.; Gomes, W.; Mouret, J.-B.; Ivaldi, S. (2020) Learning Robust Task Priorities and Gains for Control of Redundant Robots. IEEE Robotics and Automation Letters.
[3] Penco, L.; Scianca, N.; Modugno, V.; Lanari, L.; Oriolo, G.; Ivaldi, S. (2019) A Multi-Mode Teleoperation Framework for Humanoid Loco-Manipulation. IEEE Robotics & Automation Magazine. Volume 26, Issue 4, Pages 73-82. doi: 10.1109/MRA.2019.2941245
Main activities
- Write code to perform experiments with robot
- Write software documentation
- Conduct experiments in the real environment, with real robots: Tiago++
- Write scientific articles and reports
- Participate to national and European projects events (meetings, hackathons) when appropriate
Skills
Candidates should be expert in at least one of these areas:
- Robotics (kinematics, dynamics)
- Whole-body Control
- Machine learning
- Computer vision
- Modern C++ programming
- Python
- Natural language models / training
Excellent communication skills and English are required. Knowledge of the French language is not a requirement.
The ideal candidate is a team player, with good communication skills.
Other valued appreciated : proactivity, autonomy, curiosity.
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
Remuneration
2082 gross/month for the 1st and 2nd years. 2190€ gross/month for the 3rd year.
General Information
- Theme/Domain :
Robotics and Smart environments
Software engineering (BAP E) - Town/city : Villers lès Nancy
- Inria Center : Centre Inria de l'Université de Lorraine
- Starting date : 2023-10-01
- Duration of contract : 3 years
- Deadline to apply : 2023-10-31
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.
Instruction to apply
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Contacts
- Inria Team : LARSEN
-
PhD Supervisor :
Ivaldi Serena / serena.ivaldi@inria.fr
The keys to success
The ideal candidate has a MS in robotics, machine learning, artificial intelligence or computer science. He/she loves doing experiments with real robots, especially humanoids and bimanual mobile manipulators. He/she has excellent C++ and python coding skills.
The position is funded by a European project where our team is involved in a series of hackathons and experimental robot competitions, at European level. Travel in Europe and interaction with the other European teams is a fundamental part of the position.
About Inria
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.