Tracking the Cognitive Band in an Open-Ended Task.

Open-ended tasks can be decomposed into the three levels of Newell's Cognitive Band: the Unit-Task level, the Operation level, and the Deliberate-Act level. We analyzed the video game Co-op Space Fortress at these levels, reporting both the match of a cognitive model to subject behavior and the use of electroencephalogram (EEG) to track subject cognition. The Unit Task level in this game involves coordinating with a partner to kill a fortress.

Reconstructing fine-grained cognition from brain activity.

We describe the Sketch-and-Stitch method for bringing together a cognitive model and EEG to reconstruct the cognition of a subject. The method was tested in the context of a video game where the actions are highly interdependent and variable: simply changing whether a key was pressed or not for a 30th of a second can lead to a very different outcome. The Sketch level identifies the critical events in the game and the Stitch level fills in the detailed actions between these events.

Spatiotemporal analysis of event-related fMRI to reveal cognitive states.

Cognitive science has a rich history of developing theories of processing that characterize the mental steps involved in performance of many tasks. Recent work in neuroimaging and machine learning has greatly improved our ability to link cognitive processes with what is happening in the brain. This article analyzes a hidden semi-Markov model-multivoxel pattern-analysis (HSMM-MVPA) methodology that we have developed for inferring the sequence of brain states one traverses in the performance of a cognitive task.

The Common Time Course of Memory Processes Revealed.

Magnetoencephalography (MEG) was used to compare memory processes in two experiments, one involving recognition of word pairs and the other involving recall of newly learned arithmetic facts. A combination of hidden semi-Markov models and multivariate pattern analysis was used to locate brief "bumps" in the sensor data that marked the onset of different stages of cognitive processing. These bumps identified a separation between a retrieval stage that identified relevant information in memory and a decision stage that determined what response was implied by that information.

When math operations have visuospatial meanings versus purely symbolic definitions: Which solving stages and brain regions are affected?

How does processing differ during purely symbolic problem solving versus when mathematical operations can be mentally associated with meaningful (here, visuospatial) referents? Learners were trained on novel math operations (↓, ↑), that were defined strictly symbolically or in terms of a visuospatial interpretation (operands mapped to dimensions of shaded areas, answer = total area). During testing (scanner session), no visuospatial representations were displayed. However, we expected visuospatially-trained learners to form mental visuospatial representations for problems, and exhibit distinct activations.

Change-point detection of cognitive states across multiple trials in functional neuroimaging.

Many functional neuroimaging-based studies involve repetitions of a task that may require several phases, or states, of mental activity. An appealing idea is to use relevant brain regions to identify the states. We developed a novel change-point methodology that adapts to the repeated trial structure of such experiments by assuming the number of states stays fixed across similar trials while allowing the timing of change-points to change across trials.

Hidden Stages of Cognition Revealed in Patterns of Brain Activation.

To advance cognitive theory, researchers must be able to parse the performance of a task into its significant mental stages. In this article, we describe a new method that uses functional MRI brain activation to identify when participants are engaged in different cognitive stages on individual trials. The method combines multivoxel pattern analysis to identify cognitive stages and hidden semi-Markov models to identify their durations.

Phases of learning: How skill acquisition impacts cognitive processing.

This fMRI study examines the changes in participants' information processing as they repeatedly solve the same mathematical problem. We show that the majority of practice-related speedup is produced by discrete changes in cognitive processing. Because the points at which these changes take place vary from problem to problem, and the underlying information processing steps vary in duration, the existence of such discrete changes can be hard to detect.

The sequential structure of brain activation predicts skill.

In an fMRI study, participants were trained to play a complex video game. They were scanned early and then again after substantial practice. While better players showed greater activation in one region (right dorsal striatum) their relative skill was better diagnosed by considering the sequential structure of whole brain activation.

End effects and cross-dimensional interference in identification of time and length: Evidence for a common memory mechanism.

Memory plays a critical role in time estimation, yet detailed mechanisms underlying temporal memory have not been fully understood. The current functional magnetic resonance imaging (fMRI) study investigated memory phenomena in absolute identification of time durations and line lengths. In both time and length identification, participants responded faster to end-of-range stimuli (e.

Visuospatial referents facilitate the learning and transfer of mathematical operations: extending the role of the angular gyrus.

Different external representations for learning and solving mathematical operations may affect learning and transfer. To explore the effects of learning representations, learners were each introduced to two new operations (b↑n and b↓n) via either formulas or graphical representations. Both groups became adept at solving regular (trained) problems.

Extending problem-solving procedures through reflection.

A large-sample (n=75) fMRI study guided the development of a theory of how people extend their problem-solving procedures by reflecting on them. Both children and adults were trained on a new mathematical procedure and then were challenged with novel problems that required them to change and extend their procedure to solve these problems. The fMRI data were analyzed using a combination of hidden Markov models (HMMs) and multi-voxel pattern analysis (MVPA).

Discovering the structure of mathematical problem solving.

The goal of this research is to discover the stages of mathematical problem solving, the factors that influence the duration of these stages, and how these stages are related to the learning of a new mathematical competence. Using a combination of multivariate pattern analysis (MVPA) and hidden Markov models (HMM), we found that participants went through 5 major phases in solving a class of problems: A Define Phase where they identified the problem to be solved, an Encode Phase where they encoded the needed information, a Compute Phase where they performed the necessary arithmetic calculations, a Transform Phase where they performed any mathematical transformations, and a Respond Phase where they entered an answer. The Define Phase is characterized by activity in visual attention and default network regions, the Encode Phase by activity in visual regions, the Compute Phase by activity in regions active in mathematical tasks, the Transform Phase by activity in mathematical and response regions, and the Respond phase by activity in motor regions.

Detecting math problem solving strategies: an investigation into the use of retrospective self-reports, latency and fMRI data.

This research explores how to determine when mathematical problems are solved by retrieval versus computation strategies. Past research has indicated that verbal reports, solution latencies, and neural imaging all provide imperfect indicators of this distinction. Participants in the current study solved mathematical problems involving two distinct problem types, called 'Pyramid' and 'Formula' problems.

Discovering the sequential structure of thought.

Multi-voxel pattern recognition techniques combined with Hidden Markov models can be used to discover the mental states that people go through in performing a task. The combined method identifies both the mental states and how their durations vary with experimental conditions. We apply this method to a task where participants solve novel mathematical problems.

Brain networks supporting execution of mathematical skills versus acquisition of new mathematical competence.

This fMRI study examines how students extend their mathematical competence. Students solved a set of algebra-like problems. These problems included Regular Problems that have a known solution technique and Exception Problems that but did not have a known technique.

Using brain imaging to track problem solving in a complex state space.

This paper describes how behavioral and imaging data can be combined with a Hidden Markov Model (HMM) to track participants' trajectories through a complex state space. Participants completed a problem-solving variant of a memory game that involved 625 distinct states, 24 operators, and an astronomical number of paths through the state space. Three sources of information were used for classification purposes.

Tracking children's mental states while solving algebra equations.

Behavioral and function magnetic resonance imagery (fMRI) data were combined to infer the mental states of students as they interacted with an intelligent tutoring system. Sixteen children interacted with a computer tutor for solving linear equations over a six-day period (days 0-5), with days 1 and 5 occurring in an fMRI scanner. Hidden Markov model algorithms combined a model of student behavior with multi-voxel imaging pattern data to predict the mental states of students.

Brain regions engaged by part- and whole-task performance in a video game: a model-based test of the decomposition hypothesis.

Part- and whole-task conditions were created by manipulating the presence of certain components of the Space Fortress video game. A cognitive model was created for two-part games that could be combined into a model that performed the whole game. The model generated predictions both for behavioral patterns and activation patterns in various brain regions.

The neural correlates of competition during memory retrieval are modulated by attention to the cues.

As people learn more facts about a concept, those facts become more difficult to remember. This is called the fan effect, where fan refers to the number of facts known about a concept. Increasing fan has been shown to decrease accuracy and increase response time and left ventrolateral prefrontal cortex (VLPFC) activity during retrieval.

Cognitive and metacognitive activity in mathematical problem solving: prefrontal and parietal patterns.

Students were taught an algorithm for solving a new class of mathematical problems. Occasionally in the sequence of problems, they encountered exception problems that required that they extend the algorithm. Regular and exception problems were associated with different patterns of brain activation.

Neural imaging to track mental states while using an intelligent tutoring system.

Hemodynamic measures of brain activity can be used to interpret a student's mental state when they are interacting with an intelligent tutoring system. Functional magnetic resonance imaging (fMRI) data were collected while students worked with a tutoring system that taught an algebra isomorph. A cognitive model predicted the distribution of solution times from measures of problem complexity.

Lateral inferior prefrontal cortex and anterior cingulate cortex are engaged at different stages in the solution of insight problems.

Two studies used puzzles that required participants to find a word that satisfied a set of constraints. The first study used a remote-association task, where participants had to find a word that would form compound words with 3 other words. The second study required participants to complete a word fragment with an associate of another word.

Using FMRI to test models of complex cognition.

This article investigates the potential of fMRI to test assumptions about different components in models of complex cognitive tasks. If the components of a model can be associated with specific brain regions, one can make predictions for the temporal course of the BOLD response in these regions. An event-locked procedure is described for dealing with temporal variability and bringing model runs and individual data trials into alignment.

A central circuit of the mind.

The methodologies of cognitive architectures and functional magnetic resonance imaging can mutually inform each other. For example, four modules of the ACT-R (adaptive control of thought - rational) cognitive architecture have been associated with four brain regions that are active in complex tasks. Activity in a lateral inferior prefrontal region reflects retrieval of information in a declarative module; activity in a posterior parietal region reflects changes to problem representations in an imaginal module; activity in the anterior cingulate cortex reflects the updates of control information in a goal module; and activity in the caudate nucleus reflects execution of productions in a procedural module.