The overarching subject of my research is the nature of representational systems. Examples of representational systems are mathematical notations, diagrams, graphical computer interfaces and programming languages. Such symbolic systems are fundamental to human cognition and critical to the function and use of computer systems.
The particular area of my research is complex representational systems for advanced cognition in knowledge rich domains. Advanced cognition includes problem solving, conceptual learning and scientific discovery. Knowledge rich domains include topics in science and mathematics, and embraces tasks such as scheduling and process control.
I have coined the term Representational Epistemology for this area of research on representations. To obtain rich converging evidence to advance our understanding of representational epistemology, I am conducting studies on complementary aspects of complex representational systems using analytic, empirical and synthetic methods. .
See the project pages of the Representation Systems Lab for examples of this work. See my list of publications for papers.
The follow are summary statements of the areas in which I am pursuing Representational Epistemology and it applications.
This research has included:
The identification of a novel class of diagrammatic representations, which I have called Law Encoding Diagrams (LEDs), which effectively support problem solving, discovery and conceptual learning. LEDs have interesting cognitive and computational properties that contrast with other “conventional” representations and thereby provide theoretical and empirical leverage for the study of representational epistemology.
The discovery of principles for effective representational systems, which address how the conceptual structure of complex domains should be mapped to effective representational schemes.
The development of a methodological framework for the design of novel representational systems that re-codify knowledge as a means to improve complex problem solving and enhance conceptual learning.
The critical role of representations in scientific creativity has been explored by building computational models that use diagrams to make discoveries. The models demonstrate how finding the right representation was pivotal in important discoveries in the history of science and that these representations were often LEDs.
Through the design of new representational systems to serve as user interfaces, representational epistemology effects the transformation of information intensive computer-based tasks so that they are more meaningful and user centred. A past project on diagrammatic knowledge acquisition has shown how knowledge elicitation can be amplified by using graphical representations that are natural for experts, rather than relying upon notations drawn directly from AI.
An ESRC/EPSRC P@CCIT project has demonstrated that automated scheduling systems can be humanized by the design of LEDs that support users’ comprehension and interaction. The evaluations of LED-based systems for scheduling in comparison to software with conventional interfaces has shown that the knowledge, flexibility and creativity of humans and the computational power of automated systems can be used to overcome each others limitations.
Other projects are using representational epistemology to design new interfaces for the monitoring and control of complex processes; including bakery planning and scheduling, and chemical production plants.
The potential of computers to exploit and magnify the benefits of LEDs for instruction has been explored through the design and evaluation of learning environments. This work has included the design and evaluation of a number of systems:
- ReMIS-CL is a discovery learning environment for the physics of particle collisions, which exploits the LEDs used by the original scientists. The benefits to discovery the scientists accrued by inventing those LEDs appear to be conferred on students learning the same subject with those LEDs.
- AVOW-tutor is a learning environment exploiting a novel LED for electricity (see below).
- SPC-ITE is an ‘authorable’ simulation based learning environment for the graphically dominated domain of statistical process control.
Perhaps the most powerful demonstration of representational epistemology for transforming higher cognition is my work on the re-codification of knowledge in conceptually demanding domains. The theoretical principles and the methodological framework (see above) have been used to invent new LEDs for electricity (AVOW diagrams) and probability theory (PS diagrams).
The LEDs are of sufficient novelty and potential that they have been published in domain specific journals. In laboratory experiments and evaluations in real school classrooms, which compared LEDs with established notional systems, it was found that the LEDs significantly enhance conceptual learning and provided the students with powerful new problem solving strategies.
To understand the underpinning psychological basis of the representational principles, studies have been conducted to probe the memory structures, perceptual processes and task strategies involved in the use of LEDs and other graphical representations.
Using the CHREST+/EPAM cognitive architecture, a model of learning physics with LEDs has been developed that shows diagrammatic knowledge can be acquired using similar mechanisms to those found in the development of chess expertise and language acquisition. The model has a simulated eye and tests with different instructional sequences yield different organisations of chunks in long-term memory.
Using eye movement data from a task involving the interpretations of line graphs, an ACT-R/PM model has been built to explore the impact of knowledge, representational format and task type on graph-based reasoning. The model extends task analytic accounts and has demonstrated that users make a trade-off between working memory load and the complexity of task strategies.
I have developed an account of the memory structures, which are hierarchical classes of schemas, necessary for the acquisition and use of LED-based knowledge. This was done by combining verbal protocol studies of LED-based learning with the analysis of materials from the history of science.
I have recently begun to study
the nature of the process of drawing on paper and on screen. This brings
together my work in the area of learning environments and the area of underpinning
cognitive processes. We have found that there is a rich temporal signal
that appears to reveal the organization in memory of the chunks used to solve
certain graphical tasks (including drawing LEDs).
We are developing tools to extract and analyse this chunk signal. This raises the intriguing possibility that intelligent learning environments could use the chunk signal to diagnose in real time what a learner knows whilst they are solving problems. This the signal has a characteristic time scale of about 100 ms, so it has the potential to substantially increase the bandwidth of communication between the user and system.