PhD Position F/M PhD in Exascale parallel optimization using Fractal-based decomposition

This PhD will be carried out in the framework of the PEPR (Programme et Equipement Prioritaire de Recherche) NumPEx¹ project devoted to Exascale Optimization for the Exascale and financed by the France2030 investment program.

Location: INRIA Lille

Duration: 3 years

Application: Candidates must have a master in computer science or other relevant

fields. Good programming skills are required. Applications should be sent to

el-ghazali.Talbi@univ-lille.fr

They should include:

• a curriculum vitae;
• a motivation letter;
• at least two referees with their e-mail addresses;

Hosting research team: BONUS INRIA team: Solving BOPs (Big Optimization Problems) raises at least four major challenges: (1) tackling their high dimensionality; (2) handling many objectives; (3) dealing with computationally expensive objective functions; and (4) scaling on (ultra-scale) modern supercomputers. The overall scientific objectives of the Bonus project consist in addressing efficiently these challenges. On the one hand, the focus will be put on the design, analysis and implementation of optimization algorithms scalable to high-dimensional (in decision variables and/or objectives) and/or expensive problems. On the other hand, the focus will also be put on the design of optimization algorithms able to scale on heterogeneous supercomputers including several millions of processing cores. To achieve these objectives raising the associated challenges a program including three lines of research will be adopted: decomposition-based optimization, Machine Learning (ML)-assisted optimization and ultra-scale optimization.

Context

On one hand, in many scientific and industrial areas we are witnessing the emergence of big optimization problems (BOPs) which refer to problems characterised by:

• Large number of decision variables and data parameters: it will induce high-dimensionality in the problems. For instance, in smart grids, there are many BOPs for which it has to be considered a large number of consumers (e.g. appliances, electrical vehicles) and multiple suppliers.
• Mixed variables: many BOPs involve continuous, discrete and categorical variables. In the automated design of deep neural networks, the new architectures could be composed of billions of mixed variables: continuous (e.g. weights), discrete (e.g. number of layers) and categorical (e.g. type of layer).
• Expensive objective functions: optimization problems often involve time-consuming objective functions. In many BOPs, the objective functions consist in the execution of expensive simulations of a black-box complex system (e.g. multi-disciplinary engineering design).
• Multiple objective functions: indeed, optimization problems encountered in practice are rarely single-objective. In general, there are many conflicting objectives to handle; for instance, minimising the cost, maximising the performance and minimising the environment impact of a system.

Combining ultra-scale fractal decomposition and chaotic optimization for BOPs is the first break-through of this project. The main challenges are: (i) the design of chaotic global search procedures to explore the fractal space; (ii) the design of chaotic local search procedures to intensify the search in deep fractals; (iii) and the extension of chaotic search to solve multi-objective and mixed optimization problems.

On the other hand, high-performance computing (HPC) technologies have known a revolution in the last decade. HPC is evolving toward supercomputers composed of millions of cores provided by heterogeneous devices mainly multi-core CPUs with GPU accelerators. We entered the exascale era since June 2022, as the Top500 revealed the USA Frontier machine to be the first exascale supercomputer. The EU, China and Japan all have next-generation exascale projects. The EU’s ambition is to become one of the world leaders in supercomputing. France is preparing a response to EuroHPC’s next call to host the exascale European machines planned for the 2024 deadline.

Research directions

Inspired by nature-inspired complex systems, my main concern in this project is to “think” high-dimensional, massively parallel, and heterogeneous. The three innovative and complementary objectives of the project for solving BOPs in an effective and efficient way are:

• Design of ultra-scale fractal decomposition algorithms. Ultra-scale algorithms refer to algorithms generating an unprecedented and unlimited amount of independent sub-problems in parallel.
• Design of chaotic optimization algorithms for an effective and efficient search in fractal spaces.
• Heterogeneous design and implementation on Exascale supercomputers including millions of CPU and GPU cores, and billions of spiking neurons of neuromorphic architectures.

Validation and application

There is a plenty of new big optimization problems in science and industry. The proposed methodology will be validated on three complementary families of BOPs with a great environmental, societal and economic impact, and different needs in terms performance/energy trade-off. In this PhD, we will focus on the automated design of deep neural networks. Deep neural networks (DNNs), such as convolution neural networks (CNNs) and Vision Transformers (ViTs), are successful in solving various hard AI problems such as computer vision and natural language processing. DNNs are based on deep neural networks that could be giant including many layers of different types, billions of mixed variables and expensive learning [7].

References

[1] B. Mandelbroot, “The fractal geometry of nature”, W. H. Freeman and Company, 1983.

[2] A. Tsonis, “Chaos: from theory to applications”, Springer, 2012.

[3] E-G. Talbi, “Metaheuristics: from design to implementation”, Wiley, 2009.

[4] A. Di Ieva (Ed.), “The fractal geometry of the brain”, Springer, 2016.

[5] H-O. Peitgen, H. Jurgens, D. Saupe, “Chaos and fractals – new frontiers of science”, Springer, 2004.

[6] J. Michael, T. Thompson, “Chaos, fractals and their applications”, Int. Journal on Bifurcation and Chaos, Vol.26, No.13, pp.1-22, 2016.

[7] E-G. Talbi, “Automated design of deep neural networks: a survey and unified taxonomy”, ACM Computing Surveys, Vol.54, No.2, 2021.

[8] T. Skordas, “Toward a European Exascale ecosystem: The EuroHPC joint undertaking”. Communications of the ACM, Vol.62, No.2, 2019.

[9] D. Schuman et al. "Opportunities for neuromorphic computing algorithms and applications." Nature Computational Science, Vol.2, No.1, 2022

[10]  E-G. Talbi, “Machine learning into metaheuristics”, ACM Computing Surveys, Vol.54, No.6, 2022.

Technical skills and level required : computer science, parallel computing.

Languages : english

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

Gross salary per month :

• 2,100€ the first and second year
• 2,190€ for the third year