Post-Doctoral Research Visit F/M Postdoctoral position Reinforcement Learning for Collaborative Annotation

This postdoctoral position is part of the national PEPR (Programme et Equipement Prioritaire de Recherche) PlantAgroEco project, coordinated by Alexis Joly. The PEPR involves several teams from various institutes (Inria ZENITH, CIRAD AMAP,  CIRAD PHIM, CIRAD PBVMT, INRAE ePhytia, INRAE IGEPP, INRAE LISAH, IRD EGCE, IRD IEES, Univ. Paris Saclay, TelaBotanica).  This is a postdoctoral position in Machine Learning, more specifically in Reinforcement Learning. We are seeking a highly motivated and skilled postdoctoral fellow to join the project, dedicated to advancing the field of Machine Learning, with a specific focus on Reinforcement Learning. The position is initially funded for 18-month, but it can be easily extended.

The starting date is flexible, and ideally would start on Feb. 1st, 2024, but it can be earlier or later. The candidate will be based at Inria Lille - Nord Europe under the expert guidance of Odalric-Ambrym Maillard.

About Us: The PEPR PlantAgroEco project brings together multidisciplinary teams from esteemed institutes, including Inria ZENITH, CIRAD AMAP, CIRAD PHIM, and more. Our mission is to address intriguing theoretical challenges in the application of agroecological practices in agriculture through cutting-edge Machine Learning techniques.

 Collaborative Environment: You will collaborate closely with a team of dedicated Engineers responsible for the actual implementations. Hence, your primary focus will be on the creation of sound algorithms and methods, ensuring their theoretical integrity and applicability to real-world scenarios.

Odalric-Ambrym Maillard is a researcher at Inria. He has worked for over a decade on advancing the theoretical foundations of reinforcement learning,using a combination of tools from statistics, optimization and control, in order to build more efficient algorithms able to better estimate uncertainty, exploit structures, or adapt to some non-stationary context.
He was the PI of the ANR-JCJC project BADASS (BAnDits Against non-Stationarity and Structure) until Oct. 2021. He is also leading the Inria Action Exploratoire SR4SG (Sequential Recommendation for Sustainable Gardening) and the Inria-Japan associate team RELIANT (Reliable multi-armed bandits),
and is involved in a series of other projects, from more applied to more theoretical ones all related to the grand-challenge of reinforcement learning that is to make it applicable in real-life situations.
See \texttt{http://odalricambrymmaillard.neowordpress.fr} for further details.

Scool (Sequential COntinual and Online Learning) is an Inria team-project. It was created on November 1st, 2020 as the follow-up of the team SequeL. In a nutshell, the research topic of Scool is the study of the sequential decision making problem under uncertainty. Most of our activities are related to either bandit problems, or reinforcement learning problems. Through collaborations, we are working on their application in various fields, mainly: health, agriculture and ecology, sustainable development. See our \href{https://team.inria.fr/scool/projects/}{Projects page} for more information.

Your Mission: As a key member of our team, you will embark on an enriching journey to tackle complex theoretical challenges, applying your expertise to a real open-science application. This role offers a unique opportunity for a young researcher to make valuable and visible contributions in an ambitious project.

The project is organized around three high-level tasks and research questions:

1. User Annotation-Expertise Profiling: Your expertise will be instrumental in estimating and tracking user annotation profiles, adapting contextual bandit strategies to provide tailored support, and leveraging change-point detection techniques. These innovations will have wide-ranging applications beyond the scope of PlantNet, contributing to top-tier conferences and journals related to recommender systems.

2. Rapid Annotation Assistance: You will devise efficient techniques for rapid annotation, customizing approaches based on users' estimated expertise. This task involves pioneering sample-efficient hypothesis testing and personalizing assistance for optimal outcomes. Your work will provide generic-purpose approaches applicable to diverse domains.

3. Complementary Expert Query Strategies: You will pioneer adaptive query strategies for a diverse pool of experts, ensuring reliable collective labeling and adaptive stopping mechanisms. This research will not only benefit PlantNet but also have implications for other applications.

These tasks can be explored in various ways and lead to other challenges but should be considered the backbone of the project. The research, though focused on the PlantNet example, should be considered from a broader perspective, and be beneficial to recommender systems at large.

Making reinforcement learning techniques applicable to real-life applications (such as the recommendation of agroecological practices in agriculture) requires overcoming several scientific bottlenecks. Within the scope of the PEPR PlantAgroEco project, this 18m postdoc will focus on providing novel reinforcement learning strategies in order to improve the collaborative annotation process of the \href{https://plantnet.org}{PlantNet} data acquisition platform, both from a theoretical and applied perspective. This project makes appear appealing challenges around contextual multi-armed bandits relevant to collaborative decision making and recommendation at large, with a unique opportunity to interact with a real data platform used by millions. Solving the different challenges in a sound and effective way requires special attention from both mathematical and computational standpoints.

• PhD in machine learning or statistics, with a focus on multi-armed bandits or recommender systems.
• Proficiency in English.
• Strong coding abilities, coupled with analytical and statistical expertise.
• Proven background in areas such as probability, Markov chains, and concentration of measure.
• Adeptness with contextual bandits, active sampling, and recommender systems.
• Ability to work collaboratively within a dynamic scientific environment.

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage