Contract type : Public service fixed-term contract
Level of qualifications required : PhD or equivalent
Fonction : Post-Doctoral Research Visit
About the research centre or Inria department
Inria the French national institute for research in computer science and control, is dedicated to fundamental and applied research in information and communication science and technology (ICST). Inria has a workforce of 3,800 people working throughout its eight research centers established in seven regions of France.
Grenoble is the capital city of the French Alpes. Combining the urban life-style of southern France with a unique mountain setting, it is ideally situated for outdoor activities. The Grenoble area is today an important centre of industry and science (second largest in France). Dedicated to an ambitious policy in the arts, the city is host to numerous cultural institutions. With 60,000 students (including 6,000 foreign students), Grenoble is the third largest student area in France.
Reinforcement learning goal is to self-learn a task trying to maximize a reward (a game score for instance) interacting with simulations.
Recently, researchers have successfully introduced deep neural networks enabling to address more complex problems. This is often refered as
Deep Reinforcement Learning (DRL). DRL managed for instance to play many ATARI games. The most visible success of
DLR is probably AlphaGo Zero that outperformed the best human players (and itself) after being trained without using data from human games but solely through reinforcement learning. The process requires an advanced infrastructure for the training phase. For instance AlphaGo Zero trained during more than 70 hours using 64 GPU workers and19 CPU parameter servers for playing 4.9 million games of generated self-play, using 1,600 simulations for each Monte Carlo Tree Search.
The general workflow is the following. To speed up the learning process and enable a wide but thorough exploration of the parameter space, the learning neural network interacts in parallel with several instances of actors, each one consisting of a simulation of the task being learned and a neural network interacting with this simulation through the best wining strategy it knows. Periodically the actor neural networks are being updated by the learned neural network.
This workflow has evolved through various research works combining parallelization, asynchronism and novel learning strategies (GORILA, A3C, IMPALA,...).
The goal of this postdoc is to push forward the scalability of these approaches, and to proposing novel learning strategies to
learn more rapidly and more complex tasks (multiple heterogeneous tasks at once, non deterministic games, simulations of complex industrial or living systems).
This work will be performed in close collaboration in between the Sequel INRIA team specialized in DRL (https://team.inria.fr/sequel/) and the DataMove team specialized in HPC (https://team.inria.fr/datamove) .
Datamove has developed the Melissa (https://melissa-sa.github.io/) solution to manage large ensembles of parallel simulations and aggregate their data on-line in a parallel server. Melissa enabled to run thousands of simulation on up to 30 000 cores. So far Melissa was used to compute advanced statistics. But we expect this framework to be a sound base for a DRL workflow. The SequeL team has strong activities in reinforcement learning, either deep or not, ranging from theroretical aspects to applications. Among other projects, SequeL has collaborated with Mila (Montréal) to design and develop the Guesswhat?! experiment (https://guesswhat.ai/). As early as 2006, SequeL worked on go and designed the first go program (Crazy Stone) able to challenge a human expert player.
- AlphaGoZero: https://deepmind.com/blog/alphago-zero-learning-scratch/
- TensorFlow: https://www.tensorflow.org/
- Gorila https://arxiv.org/pdf/1507.04296
- A3C https://arxiv.org/abs/1602.01783
- Rainbow https://arxiv.org/abs/1710.02298
- Impala https://arxiv.org/abs/1802.01561
- Elf: https://arxiv.org/abs/1707.01067
- RAY/Rllib: https://ray.readthedocs.io/en/latest/rllib.html
- Melissa: "https://hal.inria.fr/hal-01607479v1
- Requirement: PhD in computer Science
- Location: Grenoble or Lille
- Hosting Teams:
- Sequel (INRIA Lille): https://team.inria.fr/sequel/
- DataMove (INRIA Grenoble): https://team.inria.fr/datamove
- Contact: Bruno.Raffin@inria.fr and Philippe.Preux@inria.fr
- Period: to start somewhere in 2019
- Duration: 24 months
We are looking for a candidate with a PhD either in deep learning, reinforcement learning or high performance computing (a combination of these expertise would be ideal) for a 24 month contract at INRIA. The candidate will have the possibility to join either the Sequel team at Lille or the Grenoble Team at Grenoble.
The postdoc will have access to large supercomputers equipped with multiple GPUs for experiments. We expect this work to lead to international publications sustained by advanced software prototypes.
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
Salary: 2 653 € gross/month.
Monthly salary after taxes : around 2 136,39 € (medical insurance included, income tax excluded).
- Theme/Domain :
Distributed and High Performance Computing
Statistics (Big data) (BAP E)
- Town/city : Montbonnot
- Inria Center : CRI Grenoble - Rhône-Alpes
- Starting date : 2019-09-02
- Duration of contract : 2 years
- Deadline to apply : 2019-04-30
Inria, the French national research institute for the digital sciences, promotes scientific excellence and technology transfer to maximise its impact. It employs 2,400 people. Its 200 agile project teams, generally with academic partners, involve more than 3,000 scientists in meeting the challenges of computer science and mathematics, often at the interface of other disciplines. Inria works with many companies and has assisted in the creation of over 160 startups. It strives to meet the challenges of the digital transformation of science, society and the economy.
Instruction to apply
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.