2019-02076 - R&D engineer in high-performance computing and Big Data analytics - Damaris project

Level of qualifications required : PhD or equivalent

Fonction : Temporary scientific engineer

About the research centre or Inria department

Inria, the French national research institute for the digital sciences, promotes scientific excellence and technology transfer to maximise its impact.
It employs 2,400 people. Its 200 agile project teams, generally with academic partners, involve more than 3,000 scientists in meeting the challenges of computer science and mathematics, often at the interface of other disciplines.
Inria works with many companies and has assisted in the creation of over 160 startups.
It strives to meet the challenges of the digital transformation of science, society and the economy.

Context

About Inria, the team and the position

Inria is the only French public research body fully dedicated to computational sciences. Inria's missions are to produce outstanding research in the computing and mathematical fields of digital sciences and to ensure its impact on the economy and society through technology transfer and innovation. Throughout its 8 research centres and its approximately 200 project teams, Inria has a workforce of 3 400 scientists with an annual budget of 265 million euros, 29% of which coming from its own resources. Inria Rennes Bretagne-Atlantique is one of the eight sites of Inria. This publicly funded research center has a workforce of about 620 people, including full-time research scientists, faculty staff, engineers and support staff, distributed in 33 teams and support services. 

The hired engineer will be a member of the KerData Inria team (https://team.inria.fr/kerdata/) led by Gabriel Antoniu. KerData is a joint research team of Inria Rennes - Bretagne Atlantique, ENS Rennes and INSA Rennes, and also a team of the IRISA lab. KerData's main research activities address the area of distributed data management at challenging scales, with a particular focus on clouds and petascale HPC architectures. We address the needs of data-oriented high-performance applications that must handle massive non structured data. Developed by the KerData research team in collaboration with JLESC (Joint INRIA-ANL-UIUC-BSC Lab for Extreme Scale Computing: http://publish.illinois.edu/jointlab-esc/), Damaris is a middleware for managing I/O and in situ processing of Big Data on HPC infrastructures.

Assignment

 Mission overview

 By joining our team you will participate in a dynamic work environment with exceptionally talented and friendly coworkers who are committed to high-quality research and development practices. You will collaborate with esteemed researchers from around the world by taking the technical responsibility for the development of the Damaris software, with the following global missions:

 - (1) maintain Damaris a distributable, professional-quality software (technical support, documentation, management of the web site);

 - (2) contribute to the design of a unified, scalable data processing framework where Damaris will be enhanced with innovative functionalities including Big Data storage and analytics features based on the KerA approach developed by KerData, with the goal to favor the adoption of this framework by the HPC and Big Data community, in the context of the HPC-Big Data convergence; this activity will be performed in close collaboration with our collaborators from ZettaFlow (a project aiming a startup creation focusing on these aspects).

 - (3) contribute to dissemination actions by interacting with potential users and by promoting Damaris and the future derived software through demonstrations at major HPC and Big Data events, such as the Supercomputing or IEEE BigData conferences.

 - (4) explore and lead exploitation plans, including the possibility to create a start-up company, based on Damaris, KerA and the derived software.

Main activities

Detailed missions

 Improve and extend the Damaris code, perform robustness and performance tests, maintain a continuous code integration process;

  • Create a unified data processing framework combining Damaris (as in situ/in transit data processing framework) with Big Data analytics plugins to support batch-based or stream-based data processing (e.g., based on components from the Spark, Flink ecosystem); this task includes refactoring, testing and integration with the KerA technology developed by the KerData team for low-latency stream storage and processing;
  • Adapt Damaris to use recent advanced communication and multithreading technologies (Mercury, Argobots);
  • Develop and maintain software connectors allowing Damaris to be used by state-of-the-art visualization tools such as VisIt and ParaView;
  • Facilite the dissemination of Damaris and of the unified data processing framework through the following means:

- Maintain a complete and up-to-date documentation (reference manual, user manual);

- Write example codes to facilitate learning of the interface by new users;

- Design and maintain a profesional-quality web site facilitating the distribution of the code and of its documentation;

- Design and implementation of software demonstrators: 

- Make demos at forums such as the Supercomputing conference, main international forum of the HPC community, BDEC (https://www.exascale.org/bdec/), the European Big Data Value Forum and Teratec;

- Create and animate a user community around the (maintain a mailing list, a mechanism for bug report and solving, user support, etc.).

Skills

Required qualifications

  • Excellent, demonstrated programming skills in C, C++, Python;
  • Very good knowledge of hardware and software technologies in the area of HPC (including MPI, which is a must, and knowledge about networking protocols like Intel DPDK/Infiniband)) and Big Data;
  • Very good knowledge of methodologies for managing software projects;
  • Ability to analyze and synthesize user requirements;
  • Ability to communicate and work in collaboration with experts in the same area and in other areas, in English;
  • Autonomy in leading and performing the tasks;
  • Sense of partnership and team spirit;
  • Taste for transmitting and sharing knowledge, results, progress;
  • Facility to present the results in written and oral form.

Benefits package

  • Subsidised catering service
  • Partially-reimbursed public transport

Remuneration

monthly gross salary from 2562 euros according to diploma and experience