For our next example, consider evolving the 
array of FPGA cells
shown in Figure 1. Again, the task is to be a simple
-- but non-trivial -- one, formulated to explore fundamental issues. The
circuit is to have a single input, and a single output. The input will be a
square-wave audio-tone of either 1kHz or 10kHz, and circuit is to discriminate
between them. Ideally, the output should go to a steady +5V as soon as one of
the frequencies is present, and 0V for the other one. The task was intended as
a first step into the domains of pattern recognition and signal processing,
rather than being an application in itself. One could imagine, however, such a
circuit being used to demodulate frequency-modulated binary data received over
a telephone line.
This FPGA is intended to perform digital logic, so would normally be used with a synchronising clock, as discussed in the previous section. That would make the frequency discrimination task quite straightforward: the clock could be used to time the input period. In this experiment, however, there will be no clock -- can evolution exploit the rich natural unconstrained dynamics of the silicon to achieve the task? This seems almost too much to ask: all that is available is 100 FPGA cells, each intended to perform a single Boolean logic function, and each having a delay from input to output of just a few nanoseconds (billionths of a second). How could an arbitrary structure (potentially having many recurrent -- feedback -- connections) of these 100 simple high-speed logic gates be evolved to discriminate perfectly between input periods five orders of magnitude longer than the delay through each component? Success would be significant: as well as vindicating the `unconstrained' approach to hardware evolution, the resulting circuit (requiring no external components or clock) would be incredibly efficient in its use of silicon.
The experimental arrangement is shown in Figure 8. A
genetic algorithm runs on a standard PC, and configures the real FPGA for each
fitness evaluation. The XC6216 FPGA has 
cells, so only a
corner was used. For each individual circuit, a sequence of test
tones (of 1kHz and 10kHz) were applied to the pin designated as the input, and
the signal at the pin chosen to be the output was monitored. The fitness
function was to maximise the difference in the average output voltage between
the case when the 1kHz input was present, and the case when the 10kHz was
present (see [19, 17, 6]
for full details). This average output
voltage was measured by the analogue integrator shown in the figure: the
circuit must be evaluated as a continuous-time analogue system, now we
have abandoned all of the digital design principles with which the FPGA was
intended to be used. A photograph of the circuit-board
carrying the FPGA and the circuitry used as part of the fitness measurement is
shown in Figure 9: it plugs directly into the PC, and is
simple and easily built.
Figure 8: The arrangement for the tone discriminator experiment. The
corner of cells used is shown to scale with respect to the whole
FPGA. The single input to the circuit was applied as the east-going input to a
particular cell on the west edge, as shown. The single output was designated
to be the north-going output of a particular cell on the north edge.
Figure 9: The circuitry to evolve the tone discriminator.
Throughout the experiment, an oscilloscope was directly attached to the output pin of the FPGA (see Figure 8), so that the behaviour of the evolving circuits could be visually inspected. Figure 10 shows photographs of the oscilloscope screen, illustrating the improving behaviour of the best individual in the population at various times over the course of evolution.
Figure 10: Photographs of the
oscilloscope screen. Top: the 1kHz and 10kHz input waveforms.
Below: the corresponding output of the best individual in the population
after the number of generations marked down the side.
The individual in the initial random population of 50 that happened to get the highest score produced a constant +5V output at all times, irrespective of the input. It received a fitness of slightly above zero just because of noise. Thus, there was no individual in the initial population that demonstrated any ability whatsoever to perform the task.
After 220 generations, the best circuit was basically copying the input to the output. However, on what would have been the high part of the square wave, a high frequency component was also present, visible as a blurred thickening of the line in the photograph. This high-frequency component exceeds the maximum rate at which the FPGA can make logic transitions, so the output makes small oscillations about a voltage slightly below the normal logic-high output voltage for the high part of the square wave. After another 100 generations, the behaviour was much the same, with the addition of occasional glitches to 0V when the output would otherwise have been high.
Once 650 generations had elapsed, definite progress had been made. For the 1kHz input, the output stayed high (with a small component of the input wave still present) only occasionally pulsing to a low voltage. For the 10kHz input, the input was still basically being copied to the output. By generation 1100, this behaviour had been refined, so that the output stayed almost perfectly at +5V only when the 1kHz input was present.
By generation 1400, the neat behaviour for the 1kHz input had been abandoned, but now the output was mostly high for the 1kHz input, and mostly low for the 10kHz input...with very strange looking waveforms. This behaviour was then gradually improved. Notice the waveforms at generation 2550 -- they would seem utterly absurd to a digital designer. Even though this is a digital FPGA, and we are evolving a recurrent network of logic gates, the gates are not being used to `do' logic. Logic gates are in fact high-gain arrangements of a few transistors, so that the transistors are usually saturated -- corresponding to logic 0 and 1. Evolution does not `know' that this was the intention of the designers of the FPGA, so just uses whatever behaviour these high-gain groups of transistors happen to exhibit when connected in arbitrary ways (many of which a digital designer must avoid in order to make digital logic a valid model of the system's behaviour). This is not a digital system, but a continuous-time, continuous valued dynamical system made from a recurrent arrangement of high-gain groups of transistors -- hence the unusual waveforms.
By generation 2800, the only defect in the behaviour was rapid glitching present on the output for the 10kHz input. Here, the output polarity has changed over: it is now low for the 1kHz input and high for 10kHz. Fitnesses were measured such that this swap would have no effect; in general it is a good idea to allow evolution to solve the problem in as many ways as possible -- the more solutions there are, the easier they are to find.
In the final photograph at generation 3500, we see the perfect desired behaviour. In fact, there were infrequent unwanted spikes in the output (not visible in the photograph); these were finally eliminated at around generation 4100. The GA was run for a further 1000 generations without any observable change in the behaviour of the best individual. The final circuit (which I will arbitrarily take to be the best individual of generation 5000) appears to be perfect when observed by eye on the oscilloscope. If the input is changed from 1kHz to 10kHz (or vice-versa), then the output changes cleanly between a steady +5V and a steady 0V without any perceptible delay.
The final circuit is shown in Figure 11; observe the many feedback paths. No constraining preconceptions were imposed on the circuit, so evolution was given the freedom to explore the full space of possible designs.
Figure 11: The final evolved circuit. The array of cells is shown, along with all connections that eventually
connect an output to an input. Connections driven by a cell's function output
are represented by arrows originating from the cell boundary. Connections into
a cell which are selected as inputs to its function unit have a small square
drawn on them. The actual setting of each function unit is not indicated in
this diagram.
Figure 12: The functional part of the
circuit. Cells not drawn here can be clamped to constant values without
affecting the circuit's behaviour.
By empirical testing, it was possible to determine which parts of the
array were actually contributing to the behaviour.
Figure 12 shows this functional part of the circuit. Observe
the cells shaded gray: they do influence the system's behaviour (if an
attempt is made to clamp and one of them to a constant value, then the system
malfunctions), but yet they are not connected to the main part of the circuit,
and there seems to be no route of connections by which they could ever
influence the output pin! These components must be interacting with the others
by some subtle unconventional means (such as electromagnetic coupling or
power-supply loading) which has been put to use by evolution in
composing the overall system behaviour.
By releasing the full repertoire of behaviours that the reconfigurable electronic medium can manifest, evolution has been able to craft a highly efficient complex dynamical system. Conventional design would require 1-2 orders of magnitude more silicon area to achieve the same performance with no external components or clock, and even then it would be difficult. But we have now stepped even further away from being able to understand the system in terms of familiar models. Not only do we have the rich analogue continuous-time dynamics seen in the previous section, but now the interactions between the components cannot completely be described by merely listing the wires connecting them. The functioning of the `gray cells' above shows that the interactions of the components are not solely determined by the connecting wires, but also by their positions in physical space. In particular, the spatial proximity of the components is likely to be important.
In general, the size, shape, and location of the components will be important, as well as the point-to-point connections (wires) between them. These extra means of interaction are in some ways a resource to be used, but to the extent that they are unavoidable, they could also be viewed as a constraint. These issues are crucial to understanding the evolution of physical `nervous systems', whether biological or electronic, which must necessarily exist in three-dimensional space. A particularly interesting class of spatial interactions in biology is diffuse neural messengers: although it is possible to incorporate these into an ANN model [10], the very phrase `neural network' betrays the extent to which it is often assumed that a topological network of point-to-point interconnections (of perfectly controllable strength) captures all of the important aspects of neural interaction.
I suggest that evolvable hardware, by providing a physical medium in which artificial `nervous systems' can be evolved, may provide a tool with which the evolution of natural nervous systems -- and the engineering inspiration that can be drawn from them -- may be investigated. Conversely, it is definitely the case that neuroscience is relevant to hardware evolution.