Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
About the research centre or Inria department
The Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.
Context
The design of controllers for large cyber physical systems (CPS, i.e. systems driven both by physical equations and digital controllers) is challenged today by machine learning approaches, and specifically reinforcement learning. The latter however still fail to provide guarantees on the behavior of the controllers it provides. The objective of this thesis is to explore a range of techniques that would make control design for CPS or any other large-scale complex system sound and scalable. The focus will be on quantitative methods, that provide performance guarantees, for example PAC bounds (probably approximately correct).
Several research directions are envisioned. The main one concerns model approximation. For a given dynamic system with discrete state, like a stochastic automaton, this may mean reducing the size of the state space while preserving as much as possible the distribution over generated runs, which requires computing or estimating distances between models. Starting from a CPS with continuous state variables, this means finding the best discretization with bounded state size. One may as well take as starting point a (possibly infinite) collection of representative runs of that system, or a black box trace generator, and be interested in learning a model from these traces (system identification) in order to capture the most characteristic features of their dynamics. For all these directions, one will be interested both in designing approximation algorithms, in characterizing their convergence properties, and in providing bounds for their accuracy.
A second research direction concerns approximation techniques in view of control design. There, the model (a Markov Decision Process for example) comes as the support to design an efficient control policy, toward some quantitative objective. Optimal control laws generally derive from iterative methods that do not scale up with model dimension, in particular if the latter come from discretization of continuous variables. The objective will be to explore various approximation techniques that would improve scalability, convergence speeds and provide both performance bounds and readability of the control laws. Model approximations are one possible way, but also controller regularization (for example through state aggregation), or approximations in the iterative procedure that yield optimal laws, or even control objective relaxations.
As a possible use-case for the above techniques, we aim at designing distributed controllers for large CPS, for example a fleet of trains on a subway line. The objective will be both to design multi-agent control strategies, to estimate their performances and to verify safety properties like maintaining minimal headways. Applications to other complex mechanical devices are also envisioned, like those of the OpenAi Gym.
Assignment
This PhD will take place in the SUMO Team at INRIA Rennes (Brittany, France), under the joint supervision of Loïc Hélouët and Eric Fabre. Funding is secured for this PhD, as a 3 years contract. This research will be connected to the Maveriq ANR project.
Main activities
The activities during this PhD wil consists in :
- bibliographical work,
- algorithms design and proofs,
- paper writing,
- meetings with the supervisors and with the members of the MAVERIQ ANR
- attending conferences
Skills
The candidate should be fluent in english (written, read spoken).
It is not necessary to speak french to apply.
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Possibility of teleworking ( 90 days per year) and flexible organization of working hours
- partial payment of insurance costs
Remuneration
monthly gross salary amounting to 1982 euros for the first and second years and 2085 euros for the third year
Share
General Information
- Theme/Domain : Proofs and Verification
- Town/city : Rennes
- Inria Center : CRI Rennes - Bretagne Atlantique
- Starting date : 2022-09-01
- Duration of contract : 3 years
- Deadline to apply : 2022-09-30
Contacts
- Inria Team : SUMO
-
PhD Supervisor :
Helouet Loic / loic.helouet@inria.fr
The keys to success
Candidates for this PhD should have a strong interest in formal methods. Former experience in probabilistic or quantitative model checking, or in learning techniques will be appreciated.
To get familiar with the research topic, candidates are invited to look at the SUMO team's website (http://www.irisa.fr/sumo/) and to look at the following bibliography:
Bibliography :
- “A canonical form for weighted automata and applications to approximate minimization,” Balle, Panangaden, Precup, LICS 2015
- “Adaptive state space partitioning for reinforcement learning,” Lee, Lau, Engineering Applications of Artificial Intelligence, 2004
- “Reinforcement learning with soft state aggregation,” Singh, Jaakkola, Jordan, NeurIPS 1994
- “On Satisficing in Quantitative Games,” Bansal, Chatterjee, Vardy, TACAS 2021
- “On Time with Minimal Expected Cost !” David, Jensen, Larsen et al., ATVA 2014
About Inria
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.
Instruction to apply
Please submit online : your resume, cover letter and letters of recommendation eventually
For more information, please contact loic.helouet@inria.fr
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.