PhD Position F/M Responsible Reinforcement Learning: Robustness and Privacy in and by Sequential Decision Making

The  Inria University of Lille centre, created in 2008, employs 360 people  including 305 scientists in 15 research teams. Recognised for its strong  involvement in the socio-economic development of the Hauts-De-France  region, the Inria University of Lille centre pursues a close  relationship with large companies and SMEs. By promoting synergies  between researchers and industrialists, Inria participates in the  transfer of skills and expertise in digital technologies and provides  access to the best European and international research for the benefit  of innovation and companies, particularly in the region.For more  than 10 years, the Inria University of Lille centre has been located at  the heart of Lille's university and scientific ecosystem, as well as at  the heart of Frenchtech, with a technology showroom based on Avenue de  Bretagne in Lille, on the EuraTechnologies site of economic excellence  dedicated to information and communication technologies (ICT)

In his/her journey to the doctoral thesis, the candidate will be supported by PEPR project FOUNDRY, and supervised by Debabrota Basu and Emilie Kaufmann. Debabrota and Emilie are affiliated with the Scool project-team (previously SequeL) of Inria Centre at University of Lille. As a team, Scool is internationally recognised for developing theories and algorithms for sequential learning and decision making, i.e. in the fields of bandits and reinforcement learning.

As RL algorithms are getting deployed in real-life the questions of responsible deployment, such as robustness to noise and perturbation to the feedback from environment, and privacy if users are involved in the environment yielding data.

Our works have shown that for structure-less and linear settings of multi-armed bandits and active testing (aka pure exploration) imposing privacy yields two regimes of performance. For the regime used in practice, privacy can be preserved without loss of utility. But our existing approach is not directly applicable to more practically appealing settings of RL, like MDPs or bandits with side-information (aka contexts). In these settings, there is a gap between achievable performances and the algorithms. Thus, we want to study whether the cost of privacy in contextual bandits and MDPs, and also to design optimal, computationally efficient algorithms.

Similarly, we have studied impact of unbounded corruption in feedback and safety constraints in stochastic multi-armed bandits and active testing (aka pure exploration). We want to understand how do they impact more structured RL problems and how can we design optimal algorithms in these setting.

The project is expected to simulate the existing and new collaborations with researchers and groups working on privacy-preserving machine learning, robustness, adaptive testing, and reinforcement learning. In future, the candidate will be encouraged to not only work with us but collaborate internationally. The candidate will also be part of the HumAIn alliance that aims toward studying humane impact of deploying AI.

From the application point of view, Scool is involved in multiple projects that incorporates medical data, and agricultural data. Depending on the future development, we will be interested to deploy such responsible RL systems and algorithm in such socially-meaningful applications.

This position is entirely dedicated to do one's Ph.D. thesis. French rules put a strong emphasis on the fact that the Ph.D. is completed within 3 full years of studies.

It is also possible to teach up to a reasonnable amount of time per year (say 30 hours / year to give a rough idea of what we mean by "reasonable"). More details about the topic of the Ph.D. is available at this link.

All research activities, that is bibliographical search, proposing original ideas related to the topic of the Ph.D. and developing them, presenting the work in the Scool seminar, workshops and conferences. The candidate should aim to publish the research results in premier conferences and journals of our field of research (e.g. ICML, NeurIPS, COLT, IJCAI, AAAI, JMLR). Since the work involves and impacts the responsible AI in general, the successful candidate should collaborate in writing scientific articles aiming towards the larger audience.

The candidate should preferably have the following skills:

• A strong background in mathematics/statistics
• A good knowledge of machine learning, statistics, and algorithms
• Broad interest for differential privacy and robustenss
• Knowledge of programming languages such as Python. C/C++
• Some experience with implementation and experimentation (a plus)
• A good command of English

Please follow the instructions given in https://team.inria.fr/magnet/how-to-apply/ to set up your application file.

In brief, the application of the candidate should include his/her CV, an application letter, (two or more) recommendation letters, and the school transcripts. It is reccommended that the candidate contacts Debabrota and Emilie while preparing the application.

The deadline for application is 15th July, 2024.

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

1st and 2nd year : 2100 € (gross monthly salarye)

3rd year : 2190 € (gross monthly salary)