Learning the procedure of counting represents a major step in children's development of the concept of the natural numbers. How children acquire generalized concepts of number and counting skills is still under debate. Here we investigate how a neural network agent develops representations for key concepts of counting while learning to perform several different counting tasks in a multimodal, interactive environment. We identify neural activity and connection patterns that realize a) a representation of the entity to count that was invariant to the task, b) a mapping from entity to number-word, and c) a representation of the number of entities that have been counted that was shared between tasks. The results support the notion that abstract representations of number can arise from integrating experiences across a range of number-related tasks.

Artificial Life has a long tradition of studying the interaction between learning and evolution. And, thanks to the increase in the use of individual learning techniques in Artificial Intelligence, there has been a recent revival of work combining individual and evolutionary learning. Despite the breadth of work in this area, the exact trade-offs between these two forms of learning remain unclear. In this work, we systematically examine the effect of task difficulty, the individual learning approach, and the form of inheritance on the performance of the population across different combinations of learning and evolution. We analyze in depth the conditions in which hybrid strategies that combine lifetime and evolutionary learning outperform either lifetime or evolutionary learning in isolation. We also discuss the importance of these results in both a biological and algorithmic context.

Evolutionary algorithms are powerful tools to discover novel and diverse solutions to complex problems. Here, we discuss how open-ended algorithms, such as novelty search, can be used to design and evaluate new unconventional computing systems, from the design of materials to the creation of new computational models.

In this paper, we describe and test a simple algorithm for enabling a single drone to herd a flock of birds away from a dangerous area in which birds may collide with an airplane. To do this, a drone suddenly approaches the flock from beneath, when the flock comes close to the runway. This makes the birds change their path to a higher elevation and safely pass the runway. A simulation platform for evaluating the success of different scenarios regarding this method was created based on Reynolds’ flocking rules. The results showed the possibility of this method to be successful when the drone has a climbing speed suitable for any particular flying speed of birds. In addition to flying speeds, how scattered the birds are in the flock is also another factor that was investigated in the simulations. It turned out that the success of preventing bird strikes may reduce when birds are flying at a far distance from each other.

Digital simulation enables a wide variety of research and applications underlying the study of artificial life. In evolutionary robotics applications, the focus is often on maximizing performance of an animat for a specific task. Analyzing evolved behaviors can be challenging, however, given the complex coupling of morphology and brain. In this paper, we introduce a simulation environment built to investigate animats capable of smoothly transitioning between operating modes (e.g., from cautious to aggressive or from one physical form to another). The simulator provides functionality for logging sensory information as well as animat state enabling a deep analysis. Although more abstract than soft-body or rigid-body physics engines, it is lightweight and efficient, allowing for a high number of simulations in a small amount of time. The simulation supplements other more complex physics-based environments providing for greater inspection of sensor information and animat behavior. Furthermore, it is designed to provide an extensible test bed beyond just gait transitions to assess new artificial intelligence and evolutionary algorithms and more importantly the combination of these techniques.

Learning to communicate in adaptive multi-agent populations introduces instability challenges at the individual and population levels. To develop an effective communication system, a population must converge on a shared and sufficiently stable vocabulary. We explore the factors that affect the symmetry and effectiveness of the communication protocols developed by deep reinforcement learning agents playing a coordination game. We looked at the effects of bottom-driven supervision, agent population size, and self-play (“inner speech”) on the properties of the developed communication systems. To analyse the resulting communication protocols and derive appropriate conclusions, we developed a set of information-theoretic metrics, which has been a major underdevelopment in the field. We found that all the manipulated factors greatly affect the decentralized learning outcomes of the adaptive agents. The populations with more than 2 agents or with a self-play learning mode converge on more shared and symmetric communication protocols than the 2-agent (no self-play) groups. Bottom-driven supervising feedback, in turn, augments the learning results of all groups, helping the agents learning in bigger populations or with self-play to coordinate and converge on maximally homogeneous and symmetric communication systems. We discuss the implications of our results for future work on modeling language evolution with multi-agent reinforcement learning.

Having previously developed and tested insect-inspired visual navigation algorithms for ground-based agents, we here investigate their robustness when applied to agents moving in three dimensions, to assess if they are applicable to both flying insects and robots, focusing on the impact and potential utility of changes in height. We first demonstrate that a robot implementing a route navigation algorithm can successfully navigate a route through an indoor environment at a variety of heights, even using images saved at different heights. We show that that in our environments, the efficacy of route navigation is increased with increasing height and also, for those environments, that there is better transfer of information when using images learnt at a high height to navigate when flying lower, than the other way around. This suggests that there is perhaps an adaptive value to the storing and use of views from increased height. To assess the limits to this result, we show that it is possible for a ground-based robot to recover the correct heading when using goal images stored from the perspective of a quadcopter. Through the robustness of this bio-inspired algorithm, we thus demonstrate the benefits of the ALife approach.

We train embodied agents to play Visual Hide and Seek to study the relationship between agent behaviors and environmental complexity. In Visual Hide and Seek, a prey must navigate in a simulated environment in order to avoid capture from a predator, only relying on first-person visual observations. By probing different environmental factors, agents exhibit diverse hiding strategies and even the knowledge of its own visibility to other agents in the scene. Furthermore, we quantitatively analyze how agent weaknesses, such as slower speed, affect the learned policy. Our results suggest that, although agent weakness makes the learning problem more challenging, they also cause more useful features to be learned. Our project website is available at http://www.cs.columbia.edu/bchen/visualhideseek/ .

Social feedback plays a significant role in shaping individual behavior across all types of social communities. When the social network approves or disapproves an individual's behavior, attitudes are formed at individual and group levels. In this paper, we investigate how social feedback can influence an altruistic attitude in the context of resource sharing. We use multi-agent simulations to model static and dynamic interactions through which social feedback is obtained. In particular, we examine how the structure of the interaction network can affect the attitude dynamics and the resource distribution across the group. Our results highlight the key role of network topological features such as degree, directionality and the presence of hubs. Generally, a dominant altruistic behavior leads to a more uniform resource distribution. Surprisingly, for some topologies, such as scale-free networks, individuals with the largest resource count were consistently above-average altruistic.

We can talk about learning optimisation in terms of three biological processes: evolution, development and learning. It has been argued that all three are necessary for intelligence to emerge. Together, they shape the brain through regressive and progressive plasticity. In this paper, we explored the effects of structural plasticity on learning in spiking neural networks with spike-timing-dependent plasticity: first, we systematically analysed three synapse pruning approaches (random, weight-dependent and activity-dependent) and their effects on networks’ weights, spiking activity and performance on a clustering task. Then, we examined the use of a minimalistic evolutionary approach to develop growth rules for spiking neural networks with or without pruning. We found that pruning combined with a simple weight homeostasis mechanism can be used to reduce spiking neural networks’ size without a performance loss; pruning of weak connections increases the learning rate. Evolution of developmental rules led to a rapid fitness increase of the rudimentary embryo networks; addition of pruning significantly improved the learning rate of the model, and synaptic homeostasis preserved stable spiking activity in the networks even during drastic growth.

Complex systems can exhibit autopoiesis–a remarkable capability to reproduce or restore themselves to maintain existence and functionality. We explore the resilience of autopoietic patterns–their ability to recover from shocks or perturbations–in a simplified form in Conway's Game of Life. We subject a large number of autopoietic patterns in the Game of Life to various perturbations, and record their responses using multiple resilience metrics. Our results show that while resilience is rare, we are able to identify structural features improving patterns' resilience. We also draw several parallels between the resilience of patterns in the Game of Life to real-world complex systems. Our work may be useful both for improved searching for resilient patterns in the Game of Life, and for exploring resilience in complex systems.

A regular feature of cells in most tumours is an abnormal number of chromosomes – a feature known as aneuploidy. A key mechanism towards this state is whole chromosome mis-segregation (CMS), whose role in cancer is still debated. For a long time, CMS was considered a side effect of oncogenesis, however recent research suggests instead a role as a key initiating driver of malignant transformation. Specifically, whether the mechanism of CMS can lead to the kind of mutational signature observed in early stage tumours is unknown. Furthermore, the signalling pathways themselves are still being elucidated, and the impact that these different mechanisms have on the network are yet not defined. Because of the high biological complexity, experimental limitations and overall uncertainty, ALife methods are well suited to untangle the role of CMS and shed light on its role in oncogenesis. Here we investigate the effects that CMS and point mutation have on a biologically inspired genome, implemented in silico though a gene-regulatory network (GRN) within an agent-based model (ABM). Importantly, the implementation aims to mimic real biology to facilitate possible emergent features. Each cell is equipped with chromosomes containing abstractions of key interconnected genes that are known to play a role in many cancers. We compare the effects of random mutations, where a gene is functionally altered, against CMS, where many genes are lost or gained simultaneously. Our results show that CMS is a viable mechanism for oncogenesis. Comparing CMS with the more traditional view of mutation accumulation, we show that both share similar emergent phenotypes, but that they are genotypically different. We highlight that loss of tumour suppression by either means might be the first step towards oncogenesis, and conclude that cancers probably utilize these two mechanisms in tandem. Finally, we propose that measurements of these aberrations could help to better characterize the evolution of tumours.

One goal of the Artificial Life field is to achieve a computational system with a complex richness similar to that of biological life. In lieu of the knowledge to achieve this, Open-ended evolution is often cited as a promising method. However, this is also not straightforward because it is unknown how to achieve open-ended evolution in a computational setting. One popular hypothesis is that a continuously changing fitness landscape can drive open-ended evolution toward the evolution of complex organisms. Here, we test this idea using the neuroevolution of neural network foraging agents in a smoothly and continuously changing environment for 500, 000 generations compared to an unchanging static environment. Surprisingly, we find evidence that the degree to which novel solutions are found is very similar between static and dynamic environments.

The ability to track odor plumes in dynamic environments is critical for flying insects following attractive odors to localize food or mates. This remarkable tracking behavior requires multimodal integration of odor, vision, and wind sensing, is robust to variations in plume statistics and wind speeds, and can often be performed over large distances. Therefore, it is challenging to study in confined experimental settings. Here we describe ongoing work to explore the space of policies effective to accomplish plume tracking, leveraging the reproducibility and interpretability of artificial agents trained in biologically motivated simulations. Specifically, we trained neural-network (NN) agents with deep reinforcement learning to locate the source of a patchy simulated plume, while varying their capacity to store past sensory stimuli. We analyzed the behavior of trained agents by inspecting successful trajectories. We then interrogated the input-output maps learned by the NNs, uncovering interpretable differences in control strategies introduced by varying sensory memory. We believe that our simulation-based approach can generate novel testable hypotheses to guide the development of targeted neuroethological experiments, as well as provide a pathway towards a mechanistic understanding of the key multimodal computations required for plume tracking.

Adaptation is an important capability in a fast-changing world. What factors allow an animal population to adapt to external changes in their environments? What effects do those changes have on the animal populations that do adapt? This paper explores these questions in the context of intraspecies communication in a noisy soundscape. Using a simulated soundscape and populations generated using Neuroevolution of Augmenting Topologies (NEAT), the same scenario is played through many times to understand the range of possible outcomes given an initial population and a set of noise conditions. While noise is found to have minimal effect on the best possible scenario, it affects how often that scenario is reached. The onset of noise is also found to impact the complexity of the evolved neural networks.

Cooperation among individuals has been key to sustaining societies. However, natural selection favors defection over cooperation. Cooperation can be favored when the mobility of individuals allows cooperators to form a cluster (or group). Mobility patterns of animals sometimes follow a Lévy flight. A Lévy flight is a kind of random walk but it is composed of many small movements with a few big movements. Here, we developed an agent-based model in a square lattice where agents perform Lévy flights depending on the fraction of neighboring defectors. We focus on how the sensitivity to defectors when performing Lévy flights promotes the evolution of cooperation. Results of evolutionary simulations showed that cooperation was most promoted when the sensitivity to defectors was moderate. As the population density became larger, higher sensitivity was more beneficial for cooperation to evolve.

Open-ended novelty is one of the goals of ALife. We use a recent definition of open-endedness, stated in terms of system models and meta-models, to demonstrate how the Stringmol Automata Chemistry achieves variation, innovation and emergence in a replicator-parasite system. We also show how Stringmol's self-modifying code allows certain of these novelties to be exploited within the system itself, while others are only externally observed.

In the enactivist framework, habits are precarious, self-sustaining, and self-individuating sensorimotor structures: they are a first approximation of autonomous sensorimotor entities. We present a prototype computational model which demonstrates a relatively simple mechanism for facilitating the emergence of such habits in robots. At its core this model is a system which retains a history of sensorimotor sequences which are compared and re-enacted by the controller as a function of the recent sensorimotor activity of the controlled robot. To demonstrate an application of this model concretely, we also present a minimal cognition task loosely inspired by the role of sensorimotor contingencies in human colour perception. This task requires that a robot maintains a set of particular sensorimotor coordinations which allow it to respond to different objects appropriately, influenced by an evolved behavioural history.

Mathematical disease modeling has long operated under the assumption that a single disease is caused by a single pathogen spreading among a population. This paradigm has been useful in simplifying the biological reality of contagions and has allowed the community to focus on the complexity of other factors such as population structure. However, there is an increasing amount of evidence that the strain diversity of pathogens, and their evolutionary dynamics with the host immune system, can play a large role in shaping epidemics. Here, we introduce a simple disease model with an underlying genotype network (Wagner, 2014), allowing the disease to mutate along network pathways as it spreads in a well-mixed host population. This genotype network allows us to define a genetic distance across strains and therefore model the transcendence of immunity often observed in real world pathogens (Katzelnick, 2017; Peeters, 2017). We study the emergence of epidemics in this model, or so-called epidemic phase transitions, and highlight the role of the genotype network in driving cyclicity of diseases as well as localization around key strains of the associated pathogen. More generally, our model illustrates the richness of behaviors that are possible even in well-mixed host populations once more complex genetic structures are considered to go beyond the “one disease equals one pathogen” paradigm.