MSc theses

MSc theses 2002

[2005] [2004] [2003] [2002] [2001] [2000] [1999] [1998] [1997] [< 1997]

• Shakil Afzal
  Investigation into Multiagent Systems and Reinforcement Learning
  Multi Agent System (MAS) and Reinforcement Learning is studied. The RL techniques studied are Q-Learning and a Temporal Differences (TD) method known as Adaptive Heuristic Critic or Actor-Critic. Both attempt to maximise the utility for being in a state or performing some action. The implementation of a simulator incorporating a simple grid-world is designed and implemented (using Neural Networks to represent RL utility functions). Experiments are carried out investigating these RL techniques and the effect of vicarious reinforcement on cooperation.
• Xabier Barandiaran, Full Text
  Adaptive behaviour, Autonomy and Value Systems.
  Normative function in dynamical adaptive systems
  Computational functionalism fails to understand the embodied and situated nature of behaviour by taking steady state functions as theoretical primitives, and by interpreting cognitive behaviour from a language-like, observer dependant framework without a naturalized normativity. Evolutionary functionalism, on the other hand, by grounding functional normativity on historical processes fails to give an account of normative functionality based on the present causal mechanism producing behaviour. We propose an alternative autonomous dynamical framework where functionality is defined as contribution to self-maintenance and normativity as satisfaction of closure criteria. We develop this framework by a set of formal definitions in the framework of dynamical system theory and propose the hypothesis of an homeostatic-plasticity based general purpose value system as an internalized normative mechanism that selects between internal state trajectories to produce adaptive functionality under different environmental conditions. To test the hypothesis we develop a simulation model where lower level specifications of a control arquitecture (an homeostatic plastic DRNN) give rise (through a simulated evolutionary process) to adaptive behaviour in a foraging task where food sources can be poisonous or profitable. Analysis of the evolved agent show that plastic changes occur when the agent produces salient adaptive interactions, those plastic changes determining the adaptive strategy. The embodied and interactive adaptive functionality is dynamically analysed, illustrating the autonomous dynamical framework.
  
• Matthew Bardeen, Full Text
  TD-Learning and Coevolution: Hiding or Guiding?
  
• Katie Bentley, Full Text
  Evolving Asynchronous Adaptive Systems for an Exploration of Aesthetic Pattern Formation
  This paper is an exploration of an interdisciplinary nature. Through studies in fine art, pattern formation in nature, on cellular, organism and ethological levels, and artificial life; a mechanism for a generic process of design is presented within the context of aesthetic pattern formation. Evolved random asynchronous updating schemes implemented in cellular automata and agent swarm systems with pheromonal signalling were compared favourably to deterministic and hand designed alternatives and the curious adaptive properties of the resulting evolved patterns were investigated. Aesthetic production should not be considered in isolation from aesthetic sense and thus reactions and opinions when the work was exhibited at the ICA London and Blip sci-art discussion group are included. Copious future extensions are outlined for this exciting new facet of artificial life.
• Peter Bone, Full Text
  An Evolutionary Juggling Simulator
  Juggling has long been a popular pastime for people who enjoy the challenge of developing a skill that is known to be difficult. The particular challenge of juggling is the difficulty of controlling a complex and unstable dynamic system and the opportunity to indefinitely increase the difficulty by increasing the systems complexity. The most common ways of increasing the difficulty are to increase the number of objects being juggled, juggling in more complex patterns, or performing other feats of skill and control simultaneously. The human ability of controlling dynamic systems, such as juggling, originates from the high development of close connection of sensors to actuators (hand eye co-ordination) combined with the intelligence to understand such systems - both of which have evolved over millions of years for other reasons, such as hunting. This project attempts to create a juggling simulator that learns to juggle in a similar way to a human by evolving a neural network controller. The program will learn using a genetic algorithm. A fitness function will act as a tutor to enable genes that minimize energy consumption to pass their genetic information to the next generation.
• James Casbon, Full Text
  Protein Secondary Structure Prediction with Support Vector Machines
  In this paper, a method for secondary structure with support vector machines is presented. The system used two layers of support vector machines, with a weighted cost function to balance the uneven class memberships. Using this method, prediction accuracy reaches 71.5%, comparable to the best techniques avaliable.
• William Coates, Full Text
  Exploiting Minimally Embodied Cognition
  As the embodied cognitive science perspective gains acceptance, increasingly the importance of morphology in cognition is advocated by cognitive scientists. In evolutionary robotics, this has led to a number of experiments where morphological parameters are evolved in parallel with the control structure. However, often these experiments entail highly embodied solutions, or do not explore the relative importance of embodied solutions compared to less embodied solutions. In this report a number of experiments were carried out where agents could evolve highly embodied, or less embodied solutions. In this manner we could understand in this virtual context the relative importance of embodied solutions. It turned out that agents typically used highly embodied solutions, coupled with very simple controllers, as these are generally easier to evolve. By comparing runs with evolvable morphologies and randomly initialised static morphologies, we found that evolvable morphologies did not appear to afford a significant statistical advantage. Evolvable morphological parameters appeared to result in a more robust evolutionary search, but not enough to provide a significant advantage: generally some of the randomly initialised static morphologies were adequate. However, given this results, it seems it may prove beneficial to include morphological parameters in evolutionary robotics, as at worst they will prove ineffectual: there is no evidence they hinder evolutionary search.
• Marc Cohen, Full Text
  Coevolving Poker Playing Agents
  In this project, agents are evolved to play a two player poker game (Texas Hold) using feed forward neural networks. This is done with the motivation of eventually producing as good poker players as possible. A methodology extended from previous work (Cohen, 2002) using probabilistic decision making, tournament selection, spatially oriented agents, and domain specific knowledge as inputs to the networks is presented.? It is concluded that this methodology can produce poker playing agents of a passable standard, but the addition of techniques to reduce the Red Queen effect as well as those that allow for more efficient exploration of the search space may be necessary for top level performance.
• Alice Eldridge, Full Text
  Adpative Music Systems: algorithmic process as a compositional tool
  Tracks: 01, 02, 03, 04, 05, 06, 07, 08, 09.
  Applying the Artificial Life ethos in the musical domain, the Adaptive Systems Music (AdSyM) project disregards the high level details of musical theory and attempts to generate music as it`could-be' using abstract models of adaptive systems. Three systems are built, the principal component in each being modified versions of the homeostat described by Ashby. The first 2 systems exploit the homeostatic process to generate harmonies, accompanied by a rhythm derived from the states of a one dimensional binary cellular automata. The 3rd system utilises the functional properties of the homeostat to build a prototype `trainable sequencer module'. This system allows real-time reassignment of samples to particular channels. A self-regulating homeostatic network is developed for use in the AdSyM II which is shown to preserve the essential properties of the Ashbian homeostat. Initial investigation suggests that the system may also be truly auto-regulatory. This warrants further investigation. The music generated received high commendation in public appearances, and listeners in a more rigorous evaluation conclusively supported its musicality. It is suggested that the techniques developed could be usefully applied in more theoretical areas of the field.
  
• Chrisantha Fernando, Full Text
  A Situated and Embodied Model of Classical and Instrumental Learning
  A situated and embodied evolutionary robotic model capable of classical and instrumental learning in a T-maze is evolved using a genetic algorithm. Associative learning theory is applied to analysing the behaviour. It is found that such a computational method of reverse-engineering a tightly coupled agent-environment dynamical system is inadequate for predicting the behaviour of the agent under novel environmental perturbations. However the non-robustness of the evolved model to environmental perturbations precludes making inferences about the value of such symbolic methods in describing real animal behaviour. Novel roles for rate dependent synaptic plasticity are suggested.
• Robin Freeman, Full Text
  Evolving Modularity Using Plasticity
  Modularity and plasticity are defined and their origins, identification and use in Evolutionary Robotics is examined. A hypothesis on the origin of modularity using plasticity is then presented, and an experimental framework to test this hypothesis is defined and implemented. Experiments are conducted which were initially promising but agents could not be evolved to perform the task required in the hypothesis. Further experiments are then suggested which would address this issue.
• Benoit Gaillard, Full Text
  A Neuronal Model for Visual Discrimination Tasks
  We propose a simplified model of the visual cortex, for performing a simple direction discrimination task. We use a receptive field based model for solving the correspondence problem, analyse numerically and theoretically its performance, and compare its outputs with the results presented in [9] on discrimination tasks. We redo with our model one of the experiments made by Newsome ([19]) on macaque monkeys. We compare the performance of this complete system to the human performance and to the theoretically best achievable performance.
• Carlos Gershenson, Full Text
  A Comparison of Different Cognitive Paradigms Using Simple Animats in a Virtual Laboratory, with Implications to the Notion of Cognition
  In this thesis I present a virtual laboratory which implements five different models for controlling animats: a rule-based system, a behaviour-based system, a concept-based system, a neural network, and a Braitenberg architecture. Through different experiments, I compare the performance of the models and conclude that there is no best model, since different models are better for different things in different contexts. The models I chose, although quite simple, represent different approaches for studying cognition. Using the results as an empirical philosophical aid, I note that there is no best approach for studying cognition, since different approaches have all advantages and disadvantages, because they study different aspects of cognition from different contexts. This has implications for current debates on proper approaches for cognition: all approaches are a bit proper, but none will be proper enough. I draw remarks on the notion of cognition abstracting from all the approaches used to study it, and propose a simple classification for different types of cognition.
• Eldan Goldenberg, Full Text
  Automatic layout of variable-content print data
  The increasing quantity of data held by organisations about individuals, and the recent development of digital press capable of one-off printing at a quality rivalling offset machines, have created a demand for a method to automatically generate page layouts. The present solutions to this are either to use skilled labour to hand-design each page, or constrain the design tightly by fitting everything to a template, both of which have significant drawbacks. This thesis describes a Genetic Algorithm that automatically generates page layouts, without the need for costly and time-consuming human design, and with considerably more flexibility than a template-based approach. The GA is based on related work in VLSI floorplanning, which is described and adapted for the print context. This method was found to produce attractive layouts with a relatively small number of iterations, even though the only explicit goal in the program was to minimise wasted space. Visual representations of the layout are presented and discussed, together with an analysis of the search space and the speed with which the GA finds a solution. The range of document types for which this method produces attractive layouts is considered, and finally suggestions are made for future work, which would make the system into a more complete layout generation tool. The key novel ideas in this thesis are summarised in a patent application entitled 'Page composition' submitted by Hewlett-Packard to the UK Patent Office on Friday 30th August 2002.
• Dougal Greer, Full Text
  An Explotation of Viable Architectures for Homeostatic Robot Controllers
  This report follows the development and analysis of four dynamical networks evolved for controlling a simulated robot. The controllers were evolved to perform the task of catching a series of falling balls, based on a set of experiments developed by Beer (1996) to test the cognitive ability of robot controllers. The first two networks, CTRNNs and Plastic Networks, have been used extensively in recent experiments in control from a dynamical perspective. The third, which is a homeostatic controller, is based on the work of Di Paolo (2000) and Floreano and Mondada (1998). The fourth, also a homeostatic controller, is an extension of the third but it used gas modulation Husbands (1998) to regulate learning. The homeostatic networks were found to be quick to evolve and robust. The third network was also shown to be capable of adapting to radical sensorimotor disruptions. The fourth also showed some capacity for re-adaptation to perturbations. The results of this testing raises interesting questions regarding the use of the Baldwin effect in evolving robot controllers as well as the role of the environment in adaptation, and the kinds of plastic rules that are useful for this approach to evolutionary robotics. These controllers represent first steps towards the use of homeostatic networks for an embodied, situated and dynamical approach to evolutionary robotics.
• Eleftherios Kellis, Full Text
  An Evaluation of the Scientific Potential of Evolutionary Artificial Life God-Games: Considering an Example Model for Experiments and Justification
  The mounting popularity of agent-based simulation systems triggers a number of questions concerning their potential in terms of the scientific fields they are linked with, as well as their limitations in terms of possible uses in fields related to entertainment. The realization of their aesthetic prospective has motivated the creation of models which are more than simple simulations and offer control over lower-level variables that define the system, a growing number of which should be expected to emerge over the following years. This study considers an overview of the context and principles underlying the design of autonomous agents throughout their history of existence, and proposes an example model which features a scheme for action selection inspired solely from ethological issues. Such an approach pays more attention to the potential of such techniques in terms of understanding animal behaviour, and converges to the fact that researchers should turn their heads towards real environments for the design of such agents, even though emergent behaviour is achieved with the use of a very abstract architecture. The model is subsequently aimed to incorporate features that will make it as interesting as possible, to reach the conclusion that there is adequate potential in applying agent-based simulation techniques in entertainment applications, particularly when compared with current approaches to Game Artificial Intelligence.
• Peter Law, Full Text
  A Minimally Cognitive behaviour Model of the Discrimination of Simple Size Relation
  Humans and monkeys' ability to pick out the larger or smaller of a pair of stimuli has been the subject of extensive empirical work. Results indicate that the discrimination of these and other simple relationships between stimuli is a basic competency, independent of the discrimination of the absolute qualities of the individual stimuli. The discrimination and use of these relationships by various animal species is studied by developmental and comparative psychologists, neuroscientists and ethologists. A simple, simulated artificial agent, evolved to select the larger or smaller of a pair of stimuli is presented. Part of the Minimally Cognitive behaviour research program, the model has been designed to clarify concepts and generate ideas applicable to further biological theory and research. Results illustrate a number of claims made in the biological literature: that discrimination in terms of relations is a simple and robust form of `on the spot' perceptual categorisation, and that asymmetries exist between the discrimination of `larger' and `smaller'. It is suggested that the Dynamically Recurrent Neural Network controllers used in this and many other models may themselves discriminate relationships more easily than absolute values. These results indicate that agent models of the discrimination of relationships could be a valuable tool in the scientific study of biological cognition.
• Michael Lewin, Full Text
  Concept Formation and Language Sharing: Combining Steels' Language Games with Simple Competitive Learning
  Following on from the work of Luc Steels, this paper presents a model of Concept Formation and the subsequent use of Language Games to share those concepts in a society of artificial agents. The system is naturally driven towards stability due to two important mechanisms. In the Concept Formation stage, Simple Competitive Learning results in the formation of normative categories due to the identification of naturally occurring clusters of input data. In the Language Sharing stage, the preference for words that are frequently used creates a positive feedback loop towards linguistic coherence. The model can be seen as an extension to Steels' work because it makes an important generalisation. In Steels' model, all inputs are one-dimensional and categories are formed and refined by repeatedly bisecting the space. In this model the Input Space can have any dimension and is partitioned into categories of different shapes which are not fixed. The nature of the Input Space is deliberately very general - any number of dimensions and distribution of data can be specified. Consequently the model can be applied to any categorisation situation and is relevant to both scientific research and commercial applications. This paper begins by imitating the model used by Steels and then goes on to show the importance of a ``forgetting mechanism'' in order to improve communicative success. Higher dimensional spaces are then considered and the effect of various parameters on communicative success is examined. In particular, the importance of choosing the number of objects in the Language Game correctly is explained. Finally, agents' adaptability to a foreign environment is investigated by mixing two distinct populations. /li>
• Lars Olsson, Full Text
  Anomaly Detetection Using Self/Nonself Discrimination
  In this thesis we show how computers can protect themselves from different forms of attacks, mis-configurations, and program errors. The work is inspired by the immune system and in a similar vein to the immune system our system learns how to distinguish self from nonself. The learning is done on a system call level and profiles are constructed for the analysed programs. The scheduler then decides how much processing time each process should have according to how ``normal'' the program behaves. Hence, this system can be seen as a homeostatic feedback loop where the analysis of the system calls is the sensor and the scheduler the actuator that tries to maintain a stable environment. The system is implemented as a couple of modules to the Linux kernel and analyses each system call that is made by programs added to the system. To learn and analyse profiles of the system calls we have tried three different methods, a table lookup method, a feed-forward neural network, and an Elman recurrent neural network. Experiments show that this system can detect several methods of intrusion including buffer over-flow attacks, format string attacks, and Trojan code. (Full text /li>
• Jon Palin, Full Text
  Agent-based stockmarket models: calibration issues and application
  Returns on stocks have traditionally been modelled by fitting a suitable statistical process to empirical returns. In contrast the agent-based approach to stockmarket modelling considers stock prices as arising from the interaction of a number of individual investors each trading with their own aims. This approach has been shown to reproduce features of real stockmarkets. To date the agent-based approach has been used principally as an academic research tool and not as an aid to investment. This paper considers whether a particular agent-based stockmarket model is of use in addressing the specific practical issue of how share prices respond to the sale of a large volume of shares. The paper also considers the problems faced in calibrating the model to stock returns found in real markets. /li>
• Tony Poppleton, Full Text
  Can Co-Evolution Play Ball? Competitive Co-evolution in a Pong Game
  This experiment examines co-evolution and proposes a method of achieving the theoretically-possible yet elusive 'arms-race' of continual progress. A usual stumbling block of co-evolution is usually over-specialisation to the current opponent, coupled with an inability to accurately measure the performance of the agents due to a lack of an absolute fitness measure. This experiment proposes using multiple isolated-populations to create distinct species providing varied competition against each other, in order to entice the evolution of general solutions. In addition a selection of absolute fitness metrics are evolved to be able to determine evolutionary progress. The testing ground is the game of Pong, inspired by Beer's experiments in Minimal Cognition. Agents compete against each other to put the ball past the opponent by altering the trajectory and speed of the ball. This was a different scenario to most co-evolutionary algorithms, and it is argued that the competitive symmetry in the problem allows for better analysis by avoiding additional parameters in order to ensure a fair competition. /li>
• Neil Robinson, Full Text
  Evolutionary optimisation of market-based control systems for resource allocation in computer farms
  This thesis describes the development of a market-based control (MBC) system used to allocate and balance computational tasks across a minimal simulated Utility Data Centre (UDC). Firstly, a re-implementation of the original ZIP trading-agent was developed and tested in a variety of basic markets. The re-implementation faithfully replicated the market equilibration behaviour of the original system, and a evolutionary algorithm was then employed to successfully fine-tune the performance of the system. The MBC UDC simulation is then presented as the first of its kind based on ZIP trading agents and also being fully distributed and autonomous in its operation. Experiments indicate a successful proof-of-concept and an efficient computational load balancing performance under different scenarios. Use of an evolutionary algorithm is then made to both further improve the market equilibration performance of the ZIP traders operating within the MBC simulation and to evolve the marketplaces they operate within. Thus this thesis presents the first ever (preliminary) results from using artificial evolution to automatically design the auction mechanism for an MBC system. /li>
• Darran Singleton, Full Text
  An Evolvable Approach to the Maes Action Selection Mechanism
  Action selection mechanisms typically involve considerable design effort and parameter tuning. This work considers some of the different approaches to the action selection problem, with particular attention to the Maes action selection mechanism. An implementation of the Maes mechanism is designed that requires no tuning, but instead uses a genetic algorithm to select for the most effective action selection networks from a population. In this way, a complimentary set of parameters can be evolved instead of being tuned by hand.
• Christos Skopelitis, Full Text
  Control System For A Robotic Arm
  This dissertation will present a control system for a robotic arm. A Time Delay Neural Network, which is a sub-category of dynamic neural networks, will control the arm. The aim of the project is to define a configuration for the neural network that will produce the best possible behaviour on the simple, but not trivial, task of target tracking. In order to achieve this, different methods of applying the problem will be evaluated and compared, i.e. different sensor types and network configurations. The neural network will be trained on tasks of various difficulties using two of the most widely used algorithms in Artificial Intelligence (AI), Genetic Algorithms and Backpropagation. The reader will find in this paper a brief outline of these algorithms, as well as details of the specifics of the implementation. In addition to this, the project includes a comparison of those algorithms, through the results obtained from training the neural network controlling the arm, which will demonstrate the strengths and weaknesses of their use for the problem in hand. Finally, the performance of these two algorithms will be compared with the performance of a relatively new approach, the Genetic Algorithm-Backpropagation hybrid that has recently attracted the researchers interest and has been shown to perform better than either of the two algorithms alone.
• Brian Turong, Full Text
  Trancendence: An artificial life approach to the synthesis of music
  This thesis presents results from an exploratory project to construct a system that automatically composes electronic music. The premise of the project is that composition can be viewed as a search problem with good songs being maxima in the search space of all possible songs. A Genetic Algorithm was used to compose short melodic samples and an agent-based system would mix the samples together. To aid the user in rating, a statistical approach was also used to rate the samples. Several live tests were conducted with human users and the preliminary results from those tests suggest that the statistical approach does work.
• Hywell Williams, Full Text