2022-05148 - PhD Position F/M Online Federated Learning with Non-i.i.d. Data

Contract type : Fixed-term contract

Level of qualifications required : Graduate degree or equivalent

Fonction : PhD Position

About the research centre or Inria department

The Inria research centre in Lyon (previously the Lyon branch of the Inria centre in Grenoble) is the 9th Inria research centre, formally created in December 2021.  It brings together approximately 270 people (including 110 Inria employees) in 15 research teams and research support services.

Its staff are distributed at this stage on 2 campuses: in Villeurbanne La Doua (Centre / INSA Lyon / UCBL) on the one hand, and Lyon Gerland  (ENS de Lyon) on the other. A third site should be opened in the course of 2022. The teams are mainly hosted with our partners.  The centre's teams work closely with research and higher education institutions (ENS de Lyon, UCBL, INSA Lyon, etc.), their laboratories, and other research organisations in Lyon (CNRS, INRAE, competitiveness clusters, etc.), but also with Lyon and regional economic players. Many international collaborations are also underway.

The Lyon centre is active in the fields of software, distributed and high-performance computing, embedded systems, quantum computing and privacy in the digital world, but also in digital health and computational biology.

Context

This PhD position is part of the Inria Challenge project FedMalin and will be supervised by Prof. Jean-Marie Gorce, Dr. Giovanni Neglia and Dr. Malcolm Egan. The FedMalin project focuses on the design, analysis and application of federated learning involving 10 Inria teams across 6 different Inria centers through France. The project will be carried out by 22 researchers, 16 PhD students (6 to be hired), 5 postdocs (3 to be hired) and 7 engineers (6 to be hired). Within the FedMalin project, the PhD candidate will have the opportunity to interact and collaborate with other participants exposing the candidate to theoretical, numerical and experimental aspects of federated learning.

The work will be carried out primarily within the MARACAS Inria project-team based in Lyon with co-supervision and regular research visits in the NEO project-team in Sofia-Antipolis. MARACAS and NEO have activities ranging from fundamental research to collaborations with companies. 

MARACAS is a research group consisting of approximately 15 people within Inria and INSA Lyon, this includes 6 PhD students, 2 postdocs, and 2 research engineers. The focus of MARACAS is in the theory, algorithms, and experimentation for communication systems, developing and applying methods in information theory, statistical signal processing and machine learning. The PhD position will complement existing projects on stochastic optimization and federated learning currently being carried out within MARACAS.

Assignment

The focus of the FedMalin project is on federated learning, where a number of clients with data communicate with a server in order to estimate a statistical model; e.g., a regression model based on deep neural networks. Federated learning systems support model estimation without providing the data stored in the clients directly to the server. Instead, the clients estimate local models and exchange the local model parameters or associated (sub)gradients with the server. This can dramatically reduce the amount of communication required and provide increased privacy.  

In the work carried out during this PhD, the focus is online federated learning where the data used to construct the models is not completely available initially at the clients. That is, data samples arrive while model estimation is being carried out. This is particularly relevant for crowd sensing applications, where sensors may make new observations over time. 

The goal of the PhD position is to introduce new federated stochastic approximation algorithms accounting for non i.i.d. data samples both in time and space. A convergence theory for the algorithms will be developed and validated using crowd sensing data. 

Main activities

The candidate will carry out research on federated learning algorithms in collaboration with members of the MARACAS and NEO project-teams. This includes participation in local seminars as well as in summer schools and international conferences. The candidate will also have the opportunity to do limited teaching within INSA Lyon. 

Main activities:

  • Introduce new algorithms for online federated learning.
  • Develop convergence theory for the algorithms for validation and insights into potential improvements.
  • Evaluate the algorithms on crowd sensing data.

Skills

Technical skills:

  • Background in optimization theory and probability theory/statistics; 
  • Ideally exposure to stochastic optimization algorithms and the corresponding convergence theory;
  • proficiency in python and ideally experience with machine learning packages.

The candidate must have a high level of spoken and written English.

 

Benefits package

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking (90 days / year) and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage under conditions

Remuneration

1st and 2nd year: 2051 euros gross salary /month

3rd year: 2158 euros gross salary / month