PhD Position F/M Studying the emergence of open-ended evolution in cellular automata using curiosity-driven AI (IDP 2024)

Many systems that we encounter in Nature are self-organized and dynamic, and their study often reveals the emergence of highly-structured morphologies capable of complex behaviors evolved for survival in their environment. In the artificial world, cellular automata (CAs) are among the examples of widely-studied self-organizing systems. For instance, the artificial life (ALife) community has studied the emergence of spatially localized patterns (SLPs) in CAs, giving hints to the theories of the origins of life [1]. SLPs have a local extension and can exist independently of other patterns, resembling artificial "creatures" that can survive for an extended period of time and interact with their environment. In parallel, in the embodied AI community, we generally assume an agent with a given body (morphology) and a given set of possible actions (sensorimotor capabilities), and aim to study the mechanisms of learning to control the agent behaviors (i.e. the agent's "brain"). 

In this project, we ask the following questions: how to reunite those two perspectives and jointly study the emergence of body morphologies and behavioral sensorimotor capabilities? Can we bootstrap processes of open-ended evolution in such complex systems?

To answer these questions, we will leverage recent work we did that combined 1) formalization of new classes of continuous cellular automata including mass conservation and local embedding of the CA parameters in the update dynamics [14]; 2) use of curiosity-driven exploration algorithms developed in the team to help discover novel and diverse structures in complex systems [8,15].

We believe this fundamental research project is relevant to several major scientific challenges in several disciplines. First, it addresses central open questions in the domain of artificial life, which have so far remained difficult to address due to the complexity of exploring high-dimensional self-organized systems (which here we propose to address through both new kinds of CAs with mass conservation and parameter localization, and through curiosity-driven AI for exploring the space of behaviours of these systems). Second, these questions in artificial life relate directly to fundamental questions in biology about the origins of the first evolutionary processes. Last but not least, this may set the ground for a completely different approach to building open-ended and versatile AI systems as compared to current deep learning and generative AI approaches which still either assume prior notions of agency and embodiment, or completely ignore them: here we aim to address how to build artificial systems where sensorimotor agency and simple forms of learning and evolution self-organize from scratch. 

In this project, we will consider Lenia [2,3] as an environment of study. Lenia is a system of continuous cellular automata which can generate a wide range of complex patterns and dynamics, where some of the emerging structures seem to look and behave like real-world microscopic organisms. It was developed by Bert Chan who will co-supervise this internship. 

While the notions of agents, environment, and possible agent-environment interactions are typically predefined in reinforcement learning and robotic settings; in self-organizing systems such as Lenia the notion of agent and actions (sensorimotor capabilities) is more difficult to interpret. Yet, when looking at the emergent creatures (see example video here and here), they already seem to have some sort of proto-sensorimotor control in their emergent behaviors. 

Moreover, our research team has recently proposed a new method for discovering creatures displaying sensorimotor capabilities in cellular automata [9]. For this aim, we have introduced environmental elements in Lenia to search for self-organizing creatures capable of reacting to the perturbations induced by the environment. The method is based on curriculum learning, Intrinsically Motivated Goal Exploration Processes (IMGEP, previously used for automatic scientific discovery in CAs [8]) and on gradient descent. Using a newly-introduced differentiable version of Lenia, the method is able to discover the rules leading to the emergence of robust creatures with sensorimotor capabilities. The creatures obtained, using only local update rules, are able to regenerate and preserve their integrity and structure while dealing with the obstacles or other creatures in their way. They also show great generalization to unseen environments.

The figure below shows an overvier of the last version of our Flow-Lenia system (adapted from [14]). It consists of an extension of the Lenia (a) continuous Cellular Automata (CA). Flow-Lenia (b) introduces a built-in constraint for mass conservation, strongly facilitating the discovery of life-like patterns (c), the optimization of the system parameters towards certain behaviors (d) and the introduction of environmental constraints (e). Moreover, it allows to embed the system parameters within its own local dynamics, leading to large-scale multi-species simulations analysed in the light of the Evolutionary Activity framework (f). 

The objective of this PhD thesis is to extend the range of morphological, behavioral and functional complexity discovered so far in the Lenia environment (starting from our recent works in [9 and [14]). Several directions of research will be considered during the PhD:

• Studying emergent open-ended evolution in Lenia.  An important challenge in Artificial Life and Artificial Intelligence is to design systems displaying open-ended intrinsic evolution (i.e unbounded growth of complexity through intrinsic evolutionary processes) [12]. Such a process is called intrinsic since no final objective (i.e fixed fitness function) is set by the experimenter, as in natural evolution where there is no final goal [13]. We have recently made preliminary steps in this direction by designing a CA where multiple creatures, each with their own evolvable learning rules, can co-exist and interact in a shared environment. Our first results are very promising (see last video in our paper which obtained the Best Paper Award at the Alife conference 2024 in Tokyo [14]) and we now want to scale them to larger simulations on GPU clusters to study open-ended dynamics in these systems.
• Introducing mechanisms of resource consumptions by the creatures. This will provide a more functional notion of reward for the optimization of the creatures where, instead of explicitly optimizing for moving creatures as in [9], moving behavior would emerge as a solution to collect limited resource in the environment (see [11] for an evolutionary perspective). In a second step, competition for limited resources can possibly bootstrap the emergence of species co-adaptation towards increasingly skilled creatures (a phenomenon known as “autocurricula” in the machine learning literature [10]). We have recently introduced mechanisms for mass-conservation in Lenia [14], which we think is a key step in achieving the above.
• Exploring mechanisms of reproduction among creatures. In recent experiments, we have observed such phenomena (a third creature being formed from the collusion of two others, resulting in three surviving creatures -- see [9]). Although we didn't optimize for reproduction in this prior work, the same method based on curriculum-driven IMGEP with gradient descent could be used to explicitly optimize for robust reproduction with variation, which potentially opens the road toward open-ended evolution in Lenia (in particular if coupled with the previous point).
• Studying conditions for the emergence of learning and memory mechanisms in sensorimotor creatures. This comes as a natural extension of our recent work on emergent sensorimotor behavior in [9]. We could extend the environment such that obtaining maximal reward requires creatures to encode and maintain information for some amount of time (see [11] for an evolutionary perspective). Here again, we can use the same tools (curriculum-driven IMGEP with gradient descent) to study how learning and memory can emerge from the self-organising dynamics of the creatures.

We have started several collaborations on topics related to the project, including with Bert Chan from Google Deepmind Tokyo (Japan), who is the creator of Lenia ; as well as with Michael Levin from Tufts University (USA), a renowned evolutionary and computational biologist.

We also plan to start new international collaborations in the context of this project.

We will encourage the PhD student to publish the results of the project in machine learning conference (such as NeurIPS or ICLR), in artificial life conferences (such as GECCO or ALife), or in interdisciplinary journals (such as PNAS). All travel expenses related to conferences or lab visits will be funded by Inria. All publications will be open-access on Arxiv and Hal. All source code will be open-source as well. We also encourage the publication of blog posts (our team has its own blog: https://developmentalsystems.org/)

We will provide a high-level laptop computer, a desk at Inria, funding for travel at conferences and lab visits -- as well as any equipment that might be required for the project.

The PhD student will be recruited at Inria Bordeaux and will be a member of the Flowers research team (flowers.inria.fr). The Flowers team is working on cutting-edge topics including the self-organization of behavior, large language models (such as GPT-4), deep RL, intrinsically motivated learning, automatic curriculum learning, socio-cultural interactions, developmental robotics, educational technologies. Integrating the lab is an opportunity to learn more about those topics through discussions with the team members (20 people approx. approximately a quarter of them being international). 

We have a weekly reading group and team meeting. We will encourage the PhD student to publish the results of the project in machine learning conferences (such as NeurIPS or ICLR), in artificial life conferences (such as GECCO or ALife), or in interdisciplinary journals (such as PNAS). All travel expenses related to conferences or lab visits will be funded by Inria. 

We have access to several marge-scale CPU and GPU clusters, including the Jean Zay national supercomputer. 

Inria can pay 50% of the urban commuting cost, and offer subsidized meals. 

The beach is one hour by car/bus.

References (most relevant ones are indicated with their number in bold)

[1] Randall D Beer. Autopoiesis and cognition in the game of life. Artificial Life (2004). 

[2] Bert Wang-Chak Chan. Lenia-biology of artificial life . Complex Systems (2019). 

[3] Bert Wang-Chak Chan. Lenia and expanded universe . Artificial Life (2020). https://www.mitpressjournals.org/doi/abs/10.1162/isal_a_00297 

[4] William Gilpin. Cellular automata as convolutional neural networks . Physical review (2018). 

[5] Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. Growing neural cellular automata . Distill (2020). https://distill.pub/2020/growing-ca . 

[6] Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei A Efros. Learning to control self-assembling morphologies: a study of generalization via modularity . NeurIPS (2019). 

[7] Chris Reinke, Mayalen Etcheverry and Pierre-Yves Oudeyer. Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems. ICLR (2020). Blogpost: https://developmentalsystems.org/intrinsically_motivated_discovery_of_diverse_patterns 

[8] Mayalen Etcheverry, Clément Moulin-Frier and Pierre-Yves Oudeyer. Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems . NeurIPS (2020).

[9] Gautier Hamon, Mayalen Etcheverry, Bert Chan, Clément Moulin-Frier, Pierre-Yves Oudeyer (2021). Learning Sensorimotor Agency in Cellular Automata. Blog post available at https://developmentalsystems.org/sensorimotor-lenia/ 

[10] Leibo, J. Z., Hughes, E., Lanctot, M., & Graepel, T. (2019). Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research. ArXiv Preprint ArXiv:1903.00742.

[11] Cisek, P. Resynthesizing behavior through phylogenetic refinement. Atten Percept Psychophys 81, 2265–2287 (2019). https://doi.org/10.3758/s13414-019-01760-1

[12] Stanley, K.O.: Why Open-Endedness Matters. Artificial Life 25(3), 232–235 (2019). https://doi.org/10.1162/artl_a_00294

[13] Lehman, J., Stanley, K.O.: Novelty Search and the Problem with Objectives. In: Riolo, R., Vladislavleva, E., Moore, J.H. (eds.) Genetic Programming Theory and Practice IX, pp. 37–56. Genetic and Evolutionary Computation, Springer, New York, NY (2011). https://doi.org/10.1007/978-1-4614-1770-5_3

[14] Plantec, E., Hamon, G., Etcheverry, M., Oudeyer, P. Y., Moulin-Frier, C., & Chan, B. W. C. (2023, July). Flow-Lenia: Towards open-ended evolution in cellular automata through mass conservation and parameter localization. In ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press. https://sites.google.com/view/flowlenia/

[15] Etcheverry, M. (2023). Curiosity-driven AI for Science: Automated Discovery of Self-Organized Structures (Doctoral dissertation, Inria & Labri, Université Bordeaux). https://hal.science/tel-04395833/ . Chapters 1 to 3 are the most relevant.

We are looking for motivated students having obtained a master degree in a computational domain (Computer Sciences or Physics). Strong programming skills and prior experience with Python and deep learning frameworks (preferably JAX, or Pytorch or Tensorflow) are expected. Prior experience in at least one of the folowing topics will be appreciated: numerical simulation on GPU, artificial life, artificial intelligence and machine learning, multi-agent systems, complex systems, theoretical/computational physics and biology.

• Subsidized meals
• Partial reimbursement of public transport costs
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

• 2100€ / month (before taxs) during the first 2 years,
• 2190€ / month (before taxs) during the third year