Post-Doctoral Research Visit F/M Postdoctoral position Reinforcement Learning for Collaborative Annotation

This postdoctoral position is part of the national PEPR (Programme et Equipement Prioritaire de Recherche) PlantAgroEco project, coordinated by Alexis Joly. The PEPR involves several teams from various institutes (Inria ZENITH, CIRAD AMAP,  CIRAD PHIM, CIRAD PBVMT, INRAE ePhytia, INRAE IGEPP, INRAE LISAH, IRD EGCE, IRD IEES, Univ. Paris Saclay, TelaBotanica). The position is funded for 18 months, and will be conducted a Inria Lille - Nord Europe under the supervision of Odalric-Ambrym Maillard. This is a postdoctoral position in Machine Learning, more specifically in Reinforcement Learning.

The starting date is flexible, it could start earlier than Feb. 1st, 2024.

Odalric-Ambrym Maillard is a researcher at Inria. He has worked for over a decade on advancing the theoretical foundations of reinforcement learning,
using a combination of tools from statistics, optimization and control, in order to build more efficient algorithms able to better estimate uncertainty, exploit structures, or adapt to some non-stationary context.
He was the PI of the ANR-JCJC project BADASS (BAnDits Against non-Stationarity and Structure) until Oct. 2021. He is also leading the Inria Action Exploratoire SR4SG (Sequential Recommendation for Sustainable Gardening) and the Inria-Japan associate team RELIANT (Reliable multi-armed bandits),
and is involved in a series of other projects, from more applied to more theoretical ones all related to the grand-challenge of reinforcement learning that is to make it applicable in real-life situations.
See \texttt{http://odalricambrymmaillard.neowordpress.fr} for further details.

Scool (Sequential COntinual and Online Learning) is an Inria team-project. It was created on November 1st, 2020 as the follow-up of the team SequeL. In a nutshell, the research topic of Scool is the study of the sequential decision making problem under uncertainty. Most of our activities are related to either bandit problems, or reinforcement learning problems. Through collaborations, we are working on their application in various fields, mainly: health, agriculture and ecology, sustainable development. See our \href{https://team.inria.fr/scool/projects/}{Projects page} for more information.

Topic. Making reinforcement learning techniques applicable to real-life applications (such as the recommendation of agroecological practices in agriculture) requires overcoming several scientific bottlenecks. Within the scope of the PEPR PlantAgroEco project, this 18m postdoc will focus on providing novel reinforcement learning strategies in order to improve the collaborative annotation process of the \href{https://plantnet.org}{PlantNet} data acquisition platform, both from a theoretical and applied perspective. This project makes appear appealing challenges around contextual multi-armed bandits relevant to collaborative decision making and recommendation at large, with a unique opportunity to interact with a real data platform used by millions. Solving the different challenges in a sound and effective way requires special attention from both mathematical and computational standpoints.

The project is organized around three high-level tasks and research questions:

1. The first task is about the user annotation-expertise profile (which
may vary with features and plants): Here the goal is to
estimate it, track its evolution, and improve it.,
Regarding methods, estimation could be done actively adapting contextual bandit strategies using a form of information-driven intrinsic reward, while change-point detection and expert methods are natural to help tracking. Finally, active improvement could be done via minimal interaction, active hypothesis testing
and personalized content/task recommendation.

2. The second task is to assist the users in performing rapid annotation,
using sequential hypothesis testing personalized to their (estimated) expertise.
Here pone challenge is to get rapid annotation in a possibly non-parametric context, by adapting sample efficient hypothesis testing and best-arm identification and finite-time analysis techniques.
The short number of interactions available also suggests considering a satisficing instead of optimal regret objective. Another challenge is to personalize assistance to each user expertise, which involves contextual bandit but also contextual hypothesis testing (charting) techniques.

3. A last task is to adapt query strategies of complementary experts based for the collective labeling of existing and unknown items.
One of the challenge is to handle uncertainty of experts, building adaptive confidence sets as well as sequential tests, both parametric and non-parametric, in order to perform adaptive stopping (decide when enough labeling information has been collected) in a reliable way.
Further, experts can be complementary or disagree, which wields the challenges of enforcing diversity in the pool of experts and ensuring sound collective labeling adapting majority voting systems. Last, one may consider fairness constraints on the pool of experts to avoid a large load unbalance between experts.

These tasks can be explored in various ways and lead to other challenges but should be considered the backbone of the project. The research, though focused on the PlantNet example, should be considered from a broader perspective, and be beneficial to recommender systems at large.

The postdoctoral position requires a solid capacity to code, conduct relevant numerical experiments and strong analytical skills, as well as a solid background in statistics, probability, Markov chains, concentration of measure and confidence regions, a good knowledge of multi-armed bandits, especially contextual bandits, active sampling and recommender systems processes methods, and be at ease with theoretical guarantees of the considered strategies.
The successful candidate will interact both with the Scool team at Inria Lille (specialized in bandits) and the Zenith team (hosting the PlantNet application) at Inria Montpellier and more generally with the members of the PEPR project, to produce both novel publications and modules for PlantNet (with the help of the engineers from PlantNet). A good balance between theory and application is expected throughout the project.

A PhD in machine learning or statistics, possibly related to multi-armed bandits or recommender systems.

Language: fluency in English.

Relational skills: ability to work within a group of people, listen to others, present one's work, discuss it and be able to learn from others.

While performing the assigned tasks, a certain amount of autonomy is welcome, if not necessary.

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage