Considering the fact that natural computing is multidisciplinary, made up of different fields and discipline, it is not inconceivable that there will be ‘slippery’ words or concepts, i.e. words with either totally different meaning in other fields, or with meaning that are not available, or hasn’t been agreed upon yet. The concepts are rooted in their natural plausibility and they are appealing to us for computational reasons. We derive metaphor from nature as means or inspiration which guides constructions of relevant models. An example of such a metaphor is “artificial neural network”, derived from attributing primary features from human brain to computing.

Another not so salient example is “artificial immune system” which exemplifies (not direct mapping) metaphors from nature as inspiration, i.e. computer protection against viruses. It is important to differentiate between a model and a metaphor. Models produce a quantitative representation of natural phenomena, while metaphor is more of high-level abstractions and inspiration.

It is also important to differentiate between experiment, simulation, realization and emulation.

Experiments are performed in a controlled environment, with a primary goal of collection data, or demonstration of known facts or testing hypothesis. They can be made in vivo (within living organisms) or in vitro (in an artificial environment outside a living organism). Simulations are models that are used in place of something else, with varying level of abstraction. Realization of a system involves the literal model of that system, implementing certain functionality of the original system. Emulation of a system involves reproducing its functions using another system. Natural computing approaches are aimed at simulating, emulating and sometimes realizing some natural phenomena which cleat goals.

This post will talk about these concepts. Each one of the concepts could potentially be a post of its own (… 🤔), but for the sake of brevity, the sections below will provide simplified definition as to what they mean in natural computing.

Individuals, Entities and Agents

A system, as represented in natural computing, is typically a collection of a specific unit. Think insect societies, a flock of birds, or population of humans. Individuals, entities and agents are typically used interchangeably to represent the unit component in a system. When modeling, or studying these systems, it is common to simply refer to the unit component simply as an agent. An agent could be anything between a software subroutine or an intelligent organism, such as a human.

To be considered an agent, the unit must have some degree of identity, that distinguishes it from its environment, typically in form of spatial, temporal or functional boundary. An agent should be able to interact with the said environment, as well as with other agents. As previously indicated, an agent could be of different types, i.e. biological (ants, neurons, birds, etc.), physical (robots) or virtual (computer algorithm, Pokemon etc.).

Parallelism and Distributivity

The capability of processing multiple things at the same time, parallelism, is evident in many natural phenomena such as brain processing, immune functioning and so on. Parallel processing and distributivity typically work hand in hand, as it is more comment for a natural system to process tasks in parallel across multiple agents. Most of the other concepts to be discussed later, are emergent properties of parallel and distributed operations amongst different entities.

Interactivity: Connectivity and Stigmergy

As previously indicated, a key characteristic of a natural system is that agents within the systems are able to interact amongst themselves, as well as with the environment. Interactivity is an important means nature has to generate and maintain life. Researchers have performed experiments in which some part of the gene are deleted or blocked (a process called knockout). In some of these situations, it was observed that other genes take over some or all of the deleted genes’ functions. Interactivity, therefore, leads to emergent behaviour that cannot be predicted by looking at individual components.

Connectivity is an important example of interactivity. Connectionist systems encode information throughout the nodes (individual agents), and connections between each node. In some cases, this connection also represents/quantifies the degree of interaction between nodes. Networks (of connections) are also very good examples of parallel-distributed processors, as can be observed in neural networks, as well as immune networks.

Stigmergy, on the other hand, is a form of indirect social interaction. It provides a general mechanism that reflects individual and system-level behaviour, i.e., individual behaviour modifies the environment, which in turn modifies the behaviour of other individuals. Stigmergy interactions are evident in foraging termites, where self-organization is possible as a result of the intensity of the stigmergy interactions, using a chemical substance called pheromone.

Adaptation: Learning and Evolution

Adaptation simply means the ability of a system to adjust it’s response to stimulus from its environment. Learning, the act of gaining knowledge through experience or interaction plays a key role in this adaptation process. Learning systems are those that are able to change their behaviour, and solve future information demand, based on examples (the act of learning). This invariably means learning, and hence, adapting can be regarded as changing, with the end result being knowledge acquisition.

Contrary to what might be expected, learning does not require a brain, nor is there a requirement to be conscious. This is evident in how ants forage for food, or how the immune system fight against deceases. Evolution can be regarded to be as a result of learning, although it is more characterized as an adaptive system in the context of natural computing. Theory of evolution simply means that life has changed over time. Although this idea has been in existence long before Darwin, he and scientists after him have elaborated more on it to explain the diversity of life as well as the adaptation of living things to their environment.

In contrast to learning, evolution requires specific circumstance and outcome. It requires an individual, or a group of individuals capable of reproducing with some genetic variation, followed by natural selection.

Positive and Negative Feedback

Feedback occurs when the response to a stimulus has an effect on the original stimulus. This could be negative or positive depending on if the feedback diminishes or enhances the original stimulus. The more an event triggers a positive feedback occurs, the more the event tends to occur, i.e. self-reinforcing. Negative feedback, on the other hand, plays a more regulatory role; it maintains the system equilibrium by regulating the positive feedback.

Some examples of positive feedback are human breeding, avalanche, scratching an itch, etc. While the regulatory examples of negative feedback can be observed in the ecosystem i.e. the more grasses to eat, the more rabbits reproduce till there is not enough grass for the larger rabbit population, providing some form of negative feedback for the population. Other negative feedback can be observed in human metabolism, as well as climate change impact on glaciers.

Self-Organization: Characteristics and Alternative

Self-organization is used to refer to the emergent behaviour that comes out of complex systems (natural and artificial) without any apparent instruction or blueprint. It typically involves a range of pattern formation processes in said systems. In the 19th century, queen bees used to be called kings, because it is assumed that the behaviour of the ant colony is dictated from the leader, and at the time, it was falsely assumed that only male could do anything with such organization. It is now clearly known that the primary role of the queen bee is to lay eggs, and such complex behaviour such as foraging does not require any global rules and are in fact self-organized.

Some characteristics of most self-organizing systems are:

• Collectivity and interactivity - multiple agents interacting with self and environment;
• Dynamic - continuous interactions of low-level components in order to produce and maintain structure;
• Emergent pattern - the system behaviour cannot be directly deduced from the individual agents;
• Nonlinearities - the overall state is typically larger than the sum of the individual agents;
• Complexity - they are complex by nature;
• Rule-based - typically composed of simple rules by each of the constituent agents
• Feedback loops - feedback (positive and negative) contributes to the adaptation process.

Alternatives to self-organizing systems (SOS) could either by following a leader or based a blueprint, recipe or a template.

Complexity, Emergence and Reductionism

A complex system is a system with multiple interacting agents which aggregated behaviour is more than the sum of the individual behaviours (i.e non-linear) and also exhibits some form of self-organization. A good example of a complex system is the human body, as we cannot fully deduce how it works by simply looking at the constituent organs/parts. By studying complexity, it becomes apparent that not only is the study of the internal organization of a system is necessary to fully understand said system, it is also imperative to study how the interactivity between the components of the system, as well as the environment. To experiment with complex systems, we use the previously discussed experimentation, simulation, modeling, emulation and realization.

Emergence, therefore, is the property of the whole which “emerges” but not directly possessed by the constituent parts. The neurons are the basic unit of the nervous system, but what emerges from them is more than the electrical impulses each cell (neuron) carries. What emerges from the movement of a flock of birds is a pattern that cannot be deduced from the individual movement of each bird.

The idea of emergence, however, is quite contrary to the reductionism, which used to be the position of scientists. The reductionist believes that a natural system could be fully understood by seeking out the fundamental behaviour of constituents. Although this reductionism notion is true up to a point, and indeed, it is a part of the tools to fully understand a natural phenomenon, the emergent property of these systems makes it impossible to fully conceive of the aggregate behaviour at the lower levels.

Bottom-up vs. Top-down

When pondering about how nature works we can rely heavily on computer science principles i.e. the representation of symbols and the rules/algorithms that work on said symbols. Alternatively, we can reflect on relevant natural constructs. These two approaches are the top-down and the bottom-up respectively.

In the bottom-up approach, each component’s action is based on information about its own local state, and the global behaviour emerges out of the aggregated behaviour of all participating agent. We noticed such an approach in a flock of birds or a school of ants. This is quite unlike top-down that is typically algorithmically based, and tend to mimic how human’s solve problems. Top-down approaches also assume that it is possible to fully understand a given system.

Determinism, Chaos and Fractals

Classically, a deterministic system is expected to be predicted precisely, as all events are direct consequences of enough of it’s preceding events. More recent, however, scientific developments now remodels the relations between determinism and prediction, as it turns out that there are a number of systems that can be described as deterministic, but still remains unpredictable, giving rise to what is known as the chaos theory. Chaos embodies 3 principles:

• sensitivity to initial condition;
• cause and effect are not proportional;
• non-linearities.

With the aid of mathematics, we can now calculate and describe this irregularity as a fractal, capturing the infinite complexity of the chaos theory. By fractal, it typically implies that there is a structure that is not so obvious amongst the complexity of seemingly irregular, fragmented, jagged or broken up shapes. Examples of such shapes are snowflakes, mountains, cloud e.t.c.