Analysis

A useful circuit behaviour has been evolved. The circuit has some attractive engineering properties, chiefly its small size. Its externally observable behaviour has been characterised with respect to different inputs and operating temperatures, and found to be satisfactory within particular ranges (albeit a small temperature range, and only using this particular FPGA device: see §V-A). It might be thought that this knowledge would be enough to allow the circuit to be employed confidently in an application respecting these observed limitations, but this is not so.

To advance research, or to learn new design ideas from the circuit, we need to be able to discern some of its principles of operation. However, even if the goal is merely to evolve a circuit that works (and we don't need to know how), some degree of analysis may still increase its utility as an engineering product. In particular, if bounds on possible long-term changes in the circuit's behaviour can be derived, then the circuit can be applied with confidence more widely. The need for extended consistent performance is difficult to accommodate within an evolutionary framework, because usually the tests for fitness measurement of candidate solutions (the bottleneck in the evolutionary process) are as brief as possible. The evolutionary approach can be made more viable if, through analysis, it can be predicted that a circuit will perform adequately in the long term, even though it was never tested for long during its evolution.

There are two components to long-term stability of behaviour. First, the circuit must be insensitive to certain variations in its implementation or environment: robust with respect to a necessary operational envelope. There can be a temporal aspect to robustness as it includes thermal drift over time, noise, ageing effects in semiconductor devices and integrated circuits, and so on. Second, it is possible for even simple dynamical systems to display intermittent behaviour over long timescales [60]. This is not due to any external fluctuations, but is a property of the system's own dynamics. Circuits can be constructed that will inevitably -- though after a long period of normal operation -- suddenly and unpredictably change in their qualitative mode of behaviour, possibly forever, or perhaps to return to normal operation for another long interval [61]. An evolutionary algorithm, unless using debilitatingly long fitness evaluation tests, would be blind to this pathological behaviour, and could present such a circuit as a solution to the engineering specification. Inherently erratic dynamics of this kind can also interact with the temporal aspects of the operational envelope. If analysis can provide reassurance against long-term sporadic misbehaviour, the circuit is rendered more useful.

In critical applications, complex circuits can be embedded within an error-recovering framework [62]. The error recovery mechanisms themselves can be simpler, and perhaps verified formally. For example, if a failure condition is detected, the circuit responsible could be automatically reset to a safe initial state. The more a circuit's potential failure modes are understood, the more feasible it becomes to construct a resilient system containing it.

Analysis of exotic evolved circuits is different to that undertaken as part of orthodox design. At an abstract level, the appropriate tools are sometimes more akin to neuroscience than to electronic engineering. It is especially important to recognise that an evolved system may not have a clear functional decomposition. A functional analysis decomposes the system into semi-independent subsystems with separate roles; the subsystems interact largely through their functions and independently of the details of the mechanisms that accomplish those functions [63]. Systems designed by humans can usually be understood in this way, because of the `divide and conquer' approach universally adopted to tackle complex designs.

Although an evolved system may have particular functions localised in identifiable subsystems, this is not always so. Dynamic systems theory [64] provides a mathematical framework in which systems can be characterised without a functional decomposition. Hence, what to many people is the essence of understanding -- being able to point at parts of the whole and say what function they perform -- is not always possible for evolved systems. In this case, more precisely formulated questions regarding the organisation of behaviour must replace fuzzy notions of `understanding' or `explanation' rooted in functional decomposition. In our case, these questions are centred around the suitability of an evolved circuit for engineering applications. Addressing these questions, such as those regarding long-term dynamics, is what we mean by `analysis.'

The successful action of a circuit can be considered as a property of the interface between its inner mechanisms and the external environment [63]: the inner has been adapted so that the behaviour at the interface satisfies the specification. Observations at the interface (e.g. at input and output connections) during normal circuit operation may reveal little about the inner mechanisms, but instead will largely reflect the demands of the specification. Analysis therefore requires internal probing, and/or observation under abnormal conditions, either internal or external.

There are surprisingly many tactics that can be used to piece together an analysis:

• Probing and abnormal conditions. Abnormal conditions include: manipulation of the input signals, varying temperature or power-supply voltage, replacing parts of the circuit with alternative or non-functional pieces, and injecting externally generated signals at internal points. Monitoring an internal voltage always has some side effect, often placing a mostly-reactive load at the probing point. This may have negligible consequences, but potentially perturbs the measurement, or even stops the circuit from working altogether. Probing internal signals of a circuit implemented on an FPGA can require routing extra connections to reach the external pins of the device, with a danger of further disrupting the circuit under study. Advanced non-contact measurement equipment such as the `electric-potential microscope' [65] should prove powerful for scientific understanding, but is inconvenient for regular engineering projects, requiring the surface of the silicon die to be exposed.

• Mathematical techniques, including standard electronics theory, are preferable for their rigour and generality. If a whole unconventional evolved circuit is mathematically intractable, there may still be parts of it which yield.

• Simulation of a circuit allows rapid and extensive interactive exploration. Circuits evolved not in simulation, but using real reconfigurable hardware, may rely on detailed hardware properties not easily modelled in a simulation. Attempts at simulation can at least help to clarify the extent of this dependence.

• Synthesis: a circuit can be implemented using alternative electronic devices. For example, a circuit evolved on a single VLSI reconfigurable chip, might then be constructed out of a number of hardwired small-scale chips. This provides easy access for probing and manipulation of `internal' signals, and again can clarify what aspects of the hardware are important to the circuit's operation.

• Power Consumption for the most common VLSI technology (CMOS), is related to the rate of change of the internal voltages. After removing any power-supply smoothing capacitors, power consumption can be monitored with high temporal precision, while the circuit is exposed to test conditions.

• Electromagnetic emissions, resulting from rapidly changing electrical signals, can sometimes be detected using a tuned radio receiver. Circuit activity within a chip, which might be difficult to monitor directly, can thus be roughly characterised.

• Evolutionary history: The mechanism underlying a task-achieving behaviour may be more apparent soon after its evolutionary origin, rather than after evolution has refined it closely to match the specification. It may be possible to identify the innovation (perhaps caused by one or more mutations) giving rise to the behaviour's origin in an ancestor, and to relate this to the operation of the final circuit.

• Population diversity: Sometimes there can be several slightly different (but related) forms of high-fitness circuits in an evolutionary population, which can help to reveal the basic mechanisms used.

Although unconventional evolved circuits can seem dauntingly unfamiliar, the analyst is far from powerless. We now apply these tactics to the evolved tone-discriminator, which is probably the most bizarre, mysterious, and unconventional unconstrained evolved circuit yet reported. The aim is to explore how analysis may be able to abate some of the worries associated with employing very unusual evolved circuits in an engineering application.

From the observations made so far, one could only speculate as to the circuit's means of operation, so unusual are its structure and dynamics. It was clear that continuous time played an important role. If the circuit was configured onto a different, but nominally identical, FPGA chip (not used during evolution), then its performance was degraded. If the temperature is raised or lowered, then the circuit still works, but the frequencies between which it discriminates are raised or lowered. (Digital circuits usually display unchanged behaviour followed by brittle failure on approaching their thermal limits [66].) These initial observations warranted a concerted application of our tactics for unconventional analysis.

Recall that at intermediate frequencies, the circuit's output alternates rapidly between low and high voltages (Fig. 20), otherwise it is steady high or low. This binary behaviour of the output voltage suggested that perhaps part of the system could be understood in digital terms. By temporarily making the assumption that all of the FPGA cells were acting as Boolean logic gates in the normal way, the FPGA configuration could be drawn as the logic circuit diagram of Fig. 21.

Figure 21: The logic circuit representation. The numbers in hexagons are rough estimates of time delays (in ns), based on the maximum values specified by the manufacturer. `IOB' refers to the inverting Input/Output Blocks that interface the edges of the reconfigurable array to the physical pins of the chip.

The logic circuit diagram shows several continuous-time recurrent loops (breaking the digital design rules), so the system's behaviour is unlikely to be fully captured by this Boolean level of abstraction. However, it contains many `canalysing' functions [67], such as AND and OR: functions where one input can assume a value that makes the other inputs irrelevant. It so happens that whenever our circuit's input is 1, all of the recurrent loops in Parts A & B are broken by a cascade of canalysing functions. Within 20ns of the input becoming 1, A and B satisfy the digital design rules, and all of their gates deterministically switch to fixed, static logic values, staying constant until the input next goes to 0.

Part C of the circuit is based around a 2:1 multiplexer. When Part B is in the static state, the multiplexer selects the input marked `1' to be its output. This input comes from the multiplexer's own output via an even number of inversions, resulting in no net logic inversion but a time delay of around 9ns. Under certain conditions, it is possible for such a loop to oscillate (at least transiently), but the most stable condition is for it to settle down to a constant logic state. The output of the whole circuit is taken from this loop. As this Part C loop provides the only possibility for circuit activity during a high input, the next step in the analysis was to inspect the output very carefully while applying test inputs.

We had already observed that the output only ever changes state (high$\Rightarrow$low or low$\Rightarrow$high) on the falling edge of the input waveform (Fig. 22). It was then discovered that the output also responds correctly to the width of single high-going pulses. Fig. 23 shows a low$\Rightarrow$high output transition occurring after a short pulse; further short pulses leave the output high, but a single long pulse will switch it back to the low state. The output assumes the appropriate level within 200ns after the falling edge of the input. The circuit does not respond to the width of low-going pulses, and recognises a high-going pulse delimited by as little as 200ns of low input at each end of the pulse. The output is perfectly steady at logic 1 or 0, except for a brief oscillation during the 200ns `decision time' which either dies away or results in a state change.

Figure 22: The timing of the evolved circuit's response to a change in input frequency.

Figure 23: The response to a single high-going pulse.

This is astonishing. During the single high-going pulse, we know that parts A and B of the circuit are `reset' to a static state within the first 20ns (the pulse widths are vastly longer: 500$\mu$s and 50$\mu$s correspond to 1kHz and 10kHz). Our observations at the output show that Part C is also in a static state during the pulse. Yet somehow, within 200ns of the end of the pulse, the circuit `knows' how long it was, despite being completely inactive during it.

This is hard to believe, so we have reinforced this finding through many separate types of observation, and all agree that the circuit is inactive during the pulse. Power consumption returns to quiescent levels during the pulse. Many of the internal signals were (one at a time) routed to an external pin and monitored. Sometimes this probing altered (or destroyed) the circuit's behaviour, but we have observed at least one signal from each recurrent loop while the circuit was successfully discriminating pulse-widths, and there was never activity during the pulse. We were concerned that perhaps, because of the way the gates are implemented on the FPGA, it was possible that glitches (very short-duration pulses) were able to circulate in the circuit while our logic-analysis predicts it should be static; possibly these glitches could be so short as to be unobservable when routed to an external pin. Hence, we hand-designed a high-speed `glitch-catching' circuit (basically a flip-flop) as a configuration of two FPGA cells. Glitches sufficiently strong to circulate for tens of microseconds could be expected to trigger the glitch-catcher, but it detected no activity in any of the recurrent loops during an input pulse.

The circuit is not relying on influences from outside of the chip. Once the configuration had been downloaded to the device, it could be detached from all external circuitry. The only connections to the chip were then power-supply wires from a 6V battery and shielded wires for the input and output, all directly wrapped onto the chip's pins (no socket, no decoupling capacitors). The whole isolated chip and battery assembly could then be placed in a grounded metal box, and the circuit still displayed the correct behaviour.

Figure 24: Behaviour of the first frequency-discriminating ancestor. The upper waveform is the output, and the lower the input; we see the behaviour immediately after the falling edge of a single input pulse. Left: long pulse, Right: short pulse. The input to the FPGA is actually delayed $\approx$ 40ns by an intervening buffer, relative to the wave seen here.

We performed a digital simulation of the circuit (using PSPICE), extensively exploring variations in time delays and parasitic capacitances. The simulated circuit never reproduced the FPGA configuration's successful behaviour, but did corroborate that the transient as the circuit enters its static state at the beginning of an input pulse is just a few tens of nanoseconds, in agreement with our experimental measurements of internal FPGA signals, and according with the logic analysis. We then built the circuit out of separate CMOS multiplexer chips, mimicking the way that the gates are actually implemented by multiplexers on the FPGA, and also modelling the relative time delays. Again, this circuit did not work successfully, and -- despite our best efforts -- never produced any internal activity during an input pulse.

We then went back to find the first circuit during the evolutionary run that responded at all to input frequency. Its behaviour, originally shown at generation 650 in Fig. 15 using an inadequate oscilloscope, is enlarged in Fig. 24. During a pulse, the output is steady low. After the pulse, the output oscillates at one of two different frequencies, depending on how long the preceding pulse was. These oscillations are stable and long lasting. The differences are minor between this circuit and its immediate evolutionary predecessor (which displays no discrimination, always oscillating at the lower of the two frequencies). In fact, there are no differences at all in the logic circuit diagrams; the changes do things like alter where a cell's function takes an unused input from. This lends further support that circulating glitches are not the key: there was no change to the implementation of the recurrent loops.

We see bistable oscillations similar to Fig. 24 at internal nodes of part A of the final circuit. On being released from the canalysed stable state, the difference in the first 100ns of oscillatory behaviour in part A is used by parts B & C to derive a steady output according to the pulse width. There is some initial state of the part A dynamics which is determined by the input pulse length. This initial state does not arise from any circuit activity in the normal sense: the circuit over the entire array of cells was stable and static during the pulse. It is a particular property of the FPGA implementation, as it is not reproduced in simulation or when the circuit is built from separate small chips. One guess is that the change in initial state results from some slow charge/discharge of an unknown parasitic capacitance during the pulse, but we cannot yet be sure.

We understand well how parts B & C use A's initial oscillatory dynamics to derive an orderly output, and have successfully modelled this in simulation. The time delays on the connections from A to B & C are crucial. This explains the influence of the `grey cells', which are all immediately adjacent to (or part of) the path of these signals. Varying the time delays in the simulation produces a similar result to interfering with the grey cells. Mostly, the loop of part C serves to maintain a constant and steady output even while the rest of the circuit oscillates, but immediately after an input pulse it has subtle transient dynamics interacting with those of A & B.