The Promises and Pitfalls of Virtual Reality.

Increasingly, Virtual Reality technologies are finding a place in psychology and behavioral neuroscience labs. Immersing participants in virtual worlds enables researchers to investigate empirical questions in realistic or imaginary environments while measuring a wide range of behavioral responses, without sacrificing experimental control. In this chapter, we aim to provide a balanced appraisal of VR research methods.

Launching Your VR Neuroscience Laboratory.

The proliferation and refinement of affordable virtual reality (VR) technologies and wearable sensors have opened new frontiers in cognitive and behavioral neuroscience. This chapter offers a broad overview of VR for anyone interested in leveraging it as a research tool. In the first section, it examines the fundamental functionalities of VR and outlines important considerations that inform the development of immersive content that stimulates the senses.

Event related spectral perturbations of gesture congruity: Visuospatial resources are recruited for multimodal discourse comprehension.

Here we examine the role of visuospatial working memory (WM) during the comprehension of multimodal discourse with co-speech iconic gestures. EEG was recorded as healthy adults encoded either a sequence of one (low load) or four (high load) dot locations on a grid and rehearsed them until a free recall response was collected later in the trial. During the rehearsal period of the WM task, participants observed videos of a speaker describing objects in which half of the trials included semantically related co-speech gestures (congruent), and the other half included semantically unrelated gestures (incongruent).

Verbal working memory and co-speech gesture processing.

Multimodal discourse requires an assembly of cognitive processes that are uniquely recruited for language comprehension in social contexts. In this study, we investigated the role of verbal working memory for the online integration of speech and iconic gestures. Participants memorized and rehearsed a series of auditorily presented digits in low (one digit) or high (four digits) memory load conditions.

Iconic Gestures Facilitate Discourse Comprehension in Individuals With Superior Immediate Memory for Body Configurations.

To understand a speaker's gestures, people may draw on kinesthetic working memory (KWM)-a system for temporarily remembering body movements. The present study explored whether sensitivity to gesture meaning was related to differences in KWM capacity. KWM was evaluated through sequences of novel movements that participants viewed and reproduced with their own bodies.

Neural correlates of mathematical problem solving.

This study explores electroencephalography (EEG) brain dynamics associated with mathematical problem solving. EEG and solution latencies (SLs) were recorded as 11 neurologically healthy volunteers worked on intellectually challenging math puzzles that involved combining four single-digit numbers through basic arithmetic operators (addition, subtraction, division, multiplication) to create an arithmetic expression equaling 24. Estimates of EEG spectral power were computed in three frequency bands - θ (4-7 Hz), α (8-13 Hz) and β (14-30 Hz) - over a widely distributed montage of scalp electrode sites.

Resting state and task-related brain dynamics supporting insight.

Problems can be solved in a variety of ways. One might systematically evaluate a known space of possible solutions until the right one is found. Alternatively, it may prove necessary to enlarge or restructure the expected problem space - so called "thinking outside the box.

Co-speech iconic gestures and visuo-spatial working memory.

Three experiments tested the role of verbal versus visuo-spatial working memory in the comprehension of co-speech iconic gestures. In Experiment 1, participants viewed congruent discourse primes in which the speaker's gestures matched the information conveyed by his speech, and incongruent ones in which the semantic content of the speaker's gestures diverged from that in his speech. Discourse primes were followed by picture probes that participants judged as being either related or unrelated to the preceding clip.

A psychometric measure of working memory capacity for configured body movement.

Working memory (WM) models have traditionally assumed at least two domain-specific storage systems for verbal and visuo-spatial information. We review data that suggest the existence of an additional slave system devoted to the temporary storage of body movements, and present a novel instrument for its assessment: the movement span task. The movement span task assesses individuals' ability to remember and reproduce meaningless configurations of the body.

Brain dynamics of mathematical problem solving.

The purpose of this study is to examine brain activities of participants solving mental math problems. The research investigated how problem difficulty affected the subjects' responses and electroencephalogram (EEG) in different brain regions. In general, it was found that solution latencies (SL) to the math problems increased with difficulty.

Are depictive gestures like pictures? commonalities and differences in semantic processing.

Conversation is multi-modal, involving both talk and gesture. Does understanding depictive gestures engage processes similar to those recruited in the comprehension of drawings or photographs? Event-related brain potentials (ERPs) were recorded from neurotypical adults as they viewed spontaneously produced depictive gestures preceded by congruent and incongruent contexts. Gestures were presented either dynamically in short, soundless video-clips, or statically as freeze frames extracted from gesture videos.

Gestures modulate speech processing early in utterances.

Electroencephalogram was recorded as healthy adults viewed short videos of spontaneous discourse in which a speaker used depictive gestures to complement information expressed through speech. Event-related potentials were computed time-locked to content words in the speech stream and to subsequent related and unrelated picture probes. Gestures modulated event-related potentials to content words co-timed with the first gesture in a discourse segment, relative to the same words presented with static freeze frames of the speaker.

How iconic gestures enhance communication: an ERP study.

EEG was recorded as adults watched short segments of spontaneous discourse in which the speaker's gestures and utterances contained complementary information. Videos were followed by one of four types of picture probes: cross-modal related probes were congruent with both speech and gestures; speech-only related probes were congruent with information in the speech, but not the gesture; and two sorts of unrelated probes were created by pairing each related probe with a different discourse prime. Event-related potentials (ERPs) elicited by picture probes were measured within the time windows of the N300 (250-350 ms post-stimulus) and N400 (350-550 ms post-stimulus).

Meaningful gestures: electrophysiological indices of iconic gesture comprehension.

To assess semantic processing of iconic gestures, EEG (29 scalp sites) was recorded as adults watched cartoon segments paired with soundless videos of congruous and incongruous gestures followed by probe words. Event-related potentials time-locked to the onset of gestures and probe words were measured in two experiments. In Experiment 1, participants judged the congruency between gestures and cartoons.

Right hemisphere activation of joke-related information: an event-related brain potential study.

Two studies tested the hypothesis that the right hemisphere engages in relatively coarse semantic coding that aids high-level language tasks such as joke comprehension. Scalprecorded event-related brain potentials (ERPs) were collected as healthy adults read probe words (CRAZY) preceded either by jokes or nonfunny controls ("Everyone had so much fun jumping into the swimming pool, we decided to put in a little water/platform"). Probes were related to the meaning of the jokes, but not the controls.