dbcontrol

anilseth

.

com

Modelling Natural Action Selection (CUP)

This page provides a chapter list for the CUP book, Modelling Natural Action Selection, edited by Anil Seth, Tony Prescott, and Joanna Bryson. T

[background]
[section 1]: Rational/optimal decision making
[section 2]: Computational neuroscience models
[section 3]: Social action selection

anilseth.com homepage.

Modelling Natural Action Selection

Action selection is the task of producing the right action at the right time. It involves determining available behavioral alternatives, executing those most appropriate, and resolving conflicts among competing possibilities. The aim of our proposed book is to advance the understanding of the behavioural patterns and neural substrates supporting action selection in animals, including humans. The scope of problems investigated includes:

whether biological action selection is optimal (and, if so, what is optimised),
the neural substrates for action selection in the vertebrate brain,
the role of interacting systems of perception and memory in decision making, and
the interaction of group and individual action selection.

A second aim of our book is to advance methodological practice with respect to modelling natural action selection. A wide variety of computational techniques are described, ranging from formal mathematical approaches through to computational neuroscience, connectionism and agent-based modelling. The research described has broad implications for both the natural and artificial sciences. One example is its application to medical science where models of the neural substrates for action selection are contributing to the understanding brain disorders such as Parkinson’s disease and attention deficit/hyperactivity disorder. Another is the development of novel design principles and heuristics for the construction of intelligent autonomous robots.

Return to top.

Section 1: Rational/optimal decision making

1. Introduction to the section
Seth, Prescott and Bryson

2. Do we expect natural selection to produce rational behaviour?
Alasdair I. Houston, John M. McNamara and Mark D. Steer (Bristol University)

This chapter examines how irrational behaviour can be accounted for in an evolutionary framework. The introduction to this problem explores two possibilities. One is that the outcome is a side-effect. Rules that perform well in the environment in which they evolved may not perform well in a “new” environment (e.g. the lab). Another possibility is that we misjudge what is being optimised. The settings in which ‘optimal’ decisions are investigated may be too simple, ignoring the full potential range of degrees of freedom. The chapter goes on to present an extended example, which demonstrates that violations of transitivity are possible in a state-based model if animals expect current options to be available in the future.

3. Optimal agent-based models of action selection
Anil Seth (Sussex University)

To understand what guides actionselection, it is useful to take a normative perspective that evaluates behaviour in terms of a fitness metric. This chapter describes a series of ‘artificial life’ models that bridge methods of normative modelling and individual-based or agent-based modelling. These models show that successful action selection can arise from the joint activity of parallel, loosely coupled sensori-motor processes; they also show how apparently suboptimal decision making can be accounted for by optimal foraging in competitive environments.

4. Compromise strategies for action selection
Frederick L. Crabbe (US Naval Academy)

A useful property for action selection would be the ability to select compromise actions, i.e. actions that are not the best to satisfy any active goal in isolation, but rather compromise between multiple goals. This chapter reviews the history of compromise behavior and presents computational models that quantify just how much compromise aids an agent. It demonstrates why optimal compromise behavior has a surprisingly small benefit over non-compromise behavior and suggests cases in which compromise may be truly useful.

5. Extending a biologically-inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice
Rafal Bogacz1, Marius Usher2, Jiaxiang Zhang1 and James L. McClelland3
(1Bristol University, 2Birkbeck College London, 3Carnegie Mellon University)

The Leaky Competing Accumulator (LCA) model is a biologically inspired model of choice behaviour. This chapter discusses recent analyses and extensions of the LCA model. In the first part, it reviews the values of parameters required for the LCA model to achieve optimal performance under various conditions. In the second part, the model is extended to value-based choice, where it is shown that nonlinearities in the value function, when included in the LCA model, explain some counterintuitive results from choice between alternatives characterized across multiple dimensions.

6. Bayesian approaches to modelling action selection
Konrad Körding and Max Berniker (Northwestern University)

We live in an uncertain world, and each decision may have many possible outcomes; choosing the best decision is thus complicated. This chapter describes recent research in Bayesian decision theory, which formalizes the problem of decision making in the presence of uncertainty and often provides compact models that predict observed behavior. With its elegant formalization of the problems faced by the nervous system, it promises to become a major inspiration for studies in neuroscience.

7. Sequential retrieval and inhibition of parallel (re)activated representations: A neurocomputational comparison of competitive queuing and resampling models
Eddy Davelaar (University of London, Birkbeck College)

Accumulating evidence from behavioural neuroscience indicates that performance on sequential tasks requires dedicated mechanisms for anticipating sequential structure. This chapter reviews a range of this evidence as well as several existing models of how such mechanisms might work. It then presents a decisive set of experiments favouring one account in particular, and shows how this account may also explain memory selection performance in patients with Alzheimer’s dementia and Huntington’s disease.

Return to top.

Section 2: Computational neuroscience models

8.         Introduction to the section
            Prescott, Bryson and Seth

9.         Action selection and refinement in subcortical loops through basal ganglia and cerebellum
J.C. Houk (Northwestern University)

Subcortical loops through the basal ganglia and cerebellum have powerful computational properties. This chapter presents an analysis of these properties and relates them to experimental results for two kinds of action selection: microelectrode and functional imaging data recorded during a step-tracking task, and functional imaging during a serial order recall task. The author proposes that subcortical loops through both basal ganglia and cerebellum regulate action selection. This model applies to both movement and thought, and extrapolations can account for the aetiology of schizophrenia. First, data from both monkeys and humans in a step-tracking task were used to decipher the neural mechanisms that underlie the detection of movement errors and subsequent selection of submovements that correct these errors. Second, functional imaging of human subjects during a serial order recall task was used to study brain activity associated with the selection of actions in a memorized sequence.

10. Cortical mechanisms of action selection: the affordance competition hypothesis
Paul Cisek (Montreal University)

It is commonly suggested that the brain performs action selection by first constructing representations of the world, and then by computing and executing an action plan. In contrast, this chapter proposes that the brain specifies, in parallel, several potential actions which then compete against each other while information is collected to bias that competition until a single response is selected. A computational model is described which illustrates how that competition may take place in the cerebral cortex. Simulations of the model capture qualitative features of neurophysiological data and reproduce various behavioral phenomena.

11. Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system
Thomas E. Hazy1, Michael J. Frank2 and Randall C. O'Reilly1
(1University of Colorado, 2University of Arizona)

The prefrontal cortex (PFC) has long been thought to serve as an “executive” that controls the selection of actions, and cognitive functions more generally. This chapter reviews recent attempts to elucidate the precise computational and neural mechanisms underlying the executive functions of the PFC. We suggest that the basal ganglia modulate working memory representations in prefrontal areas, to support more abstract executive functions. We have developed a computational model of this system that is capable of developing human-like performance on working memory and executive control tasks through trial-and-error learning.

12. Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy
Matthew M. Botvinick(1), Yael Niv(1), and Andrew Barto (1Princeton University, 2University of Massachussetts, Amherst)

A basic question in action selection is that of how actions are assembled into organized sequences. Most theories of routine sequential behavior assume the existence of such a strict hierarchy of internal representations, mirroring the hierarchical structure of naturalistic sequential behavior. This chapter introduces and demonstrates simulations in support of an alternative account, which asserts that the representations underlying naturalistic sequential behavior need not, and arguably cannot, assume a strictly hierarchical form.

13. Is there a brainstem substrate for action selection?
M.D. Humphries, K. Gurney and T.J. Prescott

The search for the neural substrate of vertebrate action selection has largely focused on higher brain areas such as the cortex and basal ganglia (see chapters 8-11). However, the behavioural repertoire of decerebrate and neonatal animals suggests the existence of a relatively self-contained neural substrate for action selection, in a lower part of the brain—the brainstem reticular formation. This chapter presents a novel computational model of the reticular formation and discusses its implications for understanding the overall control architecture of the vertebrate brain.

14. Understanding decision-making deficits in neurological conditions: insights from models of natural action selection
Michael Frank, Anouk Scheres, and Scott Sherman (University of Arizona)

Models of action selection implicate fronto-striatal circuits in motor control and cognitive “actions" such as decision making. Dysfunction of these circuits leads to decision-making deficits in various populations. This chapter reviews how computational models provide insights into the mechanistic basis for these deficits in Parkinson's patients, those with ventromedial frontal damage, and attention-deficit/hyperactivity disorder.

15. Biologically constrained action selection improves cognitive control in a model of the Stroop task
Tom Stafford and Kevin N. Gurney (University of Sheffield)

The Stroop task is a paradigmatic psychological task for investigating stimulus conflict and the effect this has on response selection. This chapter presents a neurocomputational model which accounts for Stroop data and which is based on the putative human locus of final response selection, the basal ganglia. This model prompts a critique of the prevailing diffusion model as a mechanism of response selection, and sheds light on the features that any response mechanism must possess to provide adaptive action selection.

16. Mechanisms of choice in the primate brain: A role for positive feedback
Jonathan Chambers, Kevin Gurney, Mark Humprhies, and Tony Prescott
(University of Sheffield)

Most accounts for neural control emphasise the widespread use of inhibition by the vertebrate brain. This chapter focuses instead on contexts in which positive feedback are necessary. It opens with a review of these conditions and also of the difficulties of regulating positive-feedback control. It presents a model of one such particular case: the primate oculomotor system, presenting it as an example of how positive feedback and competitive dynamics are used synergistically to bring about changes in gaze. This model is able to reproduce eye movement abnormalities present in sufferers of Parkinson’s disease --- a disease that affects the control of positive feedback.

Return to top.

Section 3: Social action selection

18. Introduction to the section.
Bryson, Seth and Prescott

19. Agent-based modelling as scientific method: a case study analysing primate social behaviour
Joanna J. Bryson, Yasushi Ando and Hagen Lehmann
(University of Bath)

Agent-based modelling (ABM) is becoming more prominent as a scientific methodology, yet many biologists are still uncomfortable with how such models relate to their current scientific practice. This chapter reviews the nature and limits of explanation available from ABM, and discusses how these models can be verified, validated, tested and extended. It then provides an extended case study replicating and analysing one of the best-published simulation models in biology: Hemelrijks’ account for primate social order, DomWorld. Although some aspects of DomWorld turn out not to match observed behaviour in macaque monkeys, other aspects of the model are sustained even after the faulty aspects of the model are excised.

20. An agent-based model of group decision making in baboons
W.I. Sellers1, R.A. Hill2 and B.S. Logan3
1University of Manchester, 2University of Durham, 3University of Nottingham

Agent-based modelling offers considerable scope for understanding action selection in groups.
This chapter describes an agent-based model of the key activities of a troop of baboons based on data collected at De Hoop Nature Reserve in South Africa. It analyses the predictions of the model in terms of how well it duplicates the observed activity patterns of the animals, and examines the relationship between the parameters that control the agent’s decision procedure and the model’s predictions. The results identify decisions concerning movement (group action selection) as having the greatest influence on the outcomes.

[pdf]

21. Spatial models of political competition with endogenous political parties
Michael Laver and Michel Schilperoord
(New York University)

In the social sciences, two of the most important collective action selection processes analyzed are the choice by citizens of parties to support in elections and the choice by party leaders of policy “packages” offered to citizens in order to attract this support. This chapter reviews and extends an agent-based model of party competition to deal with the birth and death of political parties. The model has been validated against real electoral and party behaviour in the Republic of Ireland. Results show that, paradoxically, the most successful vote-winning rule makes citizens on average less happy than under other policy-selection rules.

22. Optimal decision making in brains and social insect colonies
James A.R. Marshall1, Rafal Bogacz1, Anna Dornhaus2, Robert Planque3, Tim Kovacs1,and Nigel R. Franks1
(Universities of Bristol1, Arizona2, and Amsterdam3)

It is often noted that “lower” animals such as fish express social structures that resemble cognitive strategies in “higher” animals such as primates, raising the intriguing possibility of evolutionary conserved mechanisms spanning the individual and social domains. This chapter reviews newly discovered commonalities between the operations of vertebrate brains and the organisational structure of entire colonies of social insects such as ants and bees. It begins with a review of cortical theories of decision making, focusing in particular on the Usher-McClelland model (see Chapter 5). It then demonstrates how and to what extent these models can account for the behaviour of social insects. The chapter concludes with an extensive review of models of insect social behavior and recent social insect results.

23. State-dependent foraging rules for social animals in selfish herds
Sean A. Rands1,2, Richard A. Pettifor1, J. Marcus Rowcliffe1, Guy Cowlishaw1,
(Institute of Zoology, Zoological Society of London1, University of Cambridge2)

Many animals gain benefits from living in groups, such as a dilution in predation risk when they are closely aggregated (referred to as the ‘selfish herd’). Game theory has been used to predict many properties of groups (such as the expected group size), but little is known about the proximate mechanisms by which animals achieve these predicted properties. We explore a possible proximate mechanism using a spatially-explicit, individual-based model, where individuals can choose to rest or forage on the basis of a rule-of-thumb that is dependent upon both their energetic reserves and the presence and actions of neighbours. The resulting behaviour and energetic reserves of individuals, and the resulting group sizes, are shown to be affected both by the ability of the forager to detect conspecifics and areas of the environment suitable for foraging, and by the distribution of energy in the environment. The model also demonstrates that if animals are able to choose (based upon their energetic reserves) between selecting the best foraging sites available, or moving towards their neighbours for safety, then this also has significant effects upon individuals and group sizes. The implications of the proposed rule-of-thumb are discussed.

Return to top.

Return to anilseth.com homepage.