How are segmentation and binding computed and represented in the brain?

A programmatically described solution to the segmentation problem is taken as opportunity to dicuss the neural architecture problem of vision. At the center of this problem is the formation of holistic entities (the Gestalt phenomenon) out of masses of neurons (the binding problem). As formulated in the Dynamic Net Architecture (DNA), neurons can become part of a (short-term) stable state only if supported inside a coherent network ('net').

Toward understanding the neural code of the brain.

More and more, the neurosciences and the sciences concerned with mind and cognition are burying fundamental questions under layers of professional methodology. I therefore welcome Biological Cybernetics' invitation to comment on two of my papers, (von der Malsburg 1973) and (von der Malsburg and Schneider 1986) (henceforth referred to as (I) and (II)) as an opportunity to address two fundamental questions about brain and mind: How is the brain's structure generated? and How is mental content expressed by the brain's physical states? Those two questions are deeply entangled with each other and play a kind of gateway role on the way to making progress with the issues of perception, intelligence, creativity and consciousness.

An applet for the Gabor similarity scaling of the differences between complex stimuli.

It is widely accepted that after the first cortical visual area, V1, a series of stages achieves a representation of complex shapes, such as faces and objects, so that they can be understood and recognized. A major challenge for the study of complex shape perception has been the lack of a principled basis for scaling of the physical differences between stimuli so that their similarity can be specified, unconfounded by early-stage differences. Without the specification of such similarities, it is difficult to make sound inferences about the contributions of later stages to neural activity or psychophysical performance.

Self-organization of control circuits for invariant fiber projections.

Assuming that patterns in memory are represented as two-dimensional arrays of local features, just as they are in primary visual cortices, pattern recognition can take the form of elastic graph matching (Lades et al., 1993 ). Neural implementation of this may be based on preorganized fiber projections that can be activated rapidly with the help of control units (Wolfrum, Wolff, Lücke, & von der Malsburg, 2008 ).

Predicting the psychophysical similarity of faces and non-face complex shapes by image-based measures.

Shape representation is accomplished by a series of cortical stages in which cells in the first stage (V1) have local receptive fields tuned to contrast at a particular scale and orientation, each well modeled as a Gabor filter. In succeeding stages, the representation becomes largely invariant to Gabor coding (Kobatake & Tanaka, 1994). Because of the non-Gabor tuning in these later stages, which must be engaged for a behavioral response (Tong, 2003; Tong et al.

Self-organization of topographic bilinear networks for invariant recognition.

We present a model for the emergence of ordered fiber projections that may serve as a basis for invariant recognition. After invariance transformations are self-organized, so-called control units competitively activate fiber projections for different transformation parameters. The model builds on a well-known ontogenetic mechanism, activity-based development of retinotopy, and it employs activity blobs of varying position and size to install different transformations.

Experience-driven formation of parts-based representations in a model of layered visual memory.

Growing neuropsychological and neurophysiological evidence suggests that the visual cortex uses parts-based representations to encode, store and retrieve relevant objects. In such a scheme, objects are represented as a set of spatially distributed local features, or parts, arranged in stereotypical fashion. To encode the local appearance and to represent the relations between the constituent parts, there has to be an appropriate memory structure formed by previous experience with visual objects.

A recurrent dynamic model for correspondence-based face recognition.

Our aim here is to create a fully neural, functionally competitive, and correspondence-based model for invariant face recognition. By recurrently integrating information about feature similarities, spatial feature relations, and facial structure stored in memory, the system evaluates face identity ("what"-information) and face position ("where"-information) using explicit representations for both. The network consists of three functional layers of processing, (1) an input layer for image representation, (2) a middle layer for recurrent information integration, and (3) a gallery layer for memory storage.

Steps toward numerical mode analysis of organizing systems.

A well established method to analyze dynamical systems described by coupled nonlinear differential equations is to determine their normal modes and reduce the dynamics, by adiabatic elimination of stable modes, to a much smaller system for the amplitudes of unstable modes and their nonlinear interactions. So far, this analysis is possible only for idealized symmetric model systems. We aim to build a framework in which realistic systems with less symmetry can be analyzed automatically.

Rapid convergence to feature layer correspondences.

We describe a neural network able to rapidly establish correspondence between neural feature layers. Each of the network's two layers consists of interconnected cortical columns, and each column consists of inhibitorily coupled subpopulations of excitatory neurons. The dynamics of the system builds on a dynamic model of a single column, which is consistent with recent experimental findings.

Figure-ground separation by cue integration.

This letter presents an improved cue integration approach to reliably separate coherent moving objects from their background scene in video sequences. The proposed method uses a probabilistic framework to unify bottom-up and top-down cues in a parallel, "democratic" fashion. The algorithm makes use of a modified Bayes rule where each pixel's posterior probabilities of figure or ground layer assignment are derived from likelihood models of three bottom-up cues and a prior model provided by a top-down cue.

Establishment of a scaffold for orientation maps in primary visual cortex of higher mammals.

In higher mammals, environmentally driven patterns of neural activity do not play a role in the establishment of orientation specificity and maps. It has been proposed that specific long-range interactions provide the scaffold for developing orientation maps. Our model aims at explaining how such a scaffold could develop in the first place.

What is the optimal architecture for visual information routing?

Analyzing the design of networks for visual information routing is an underconstrained problem due to insufficient anatomical and physiological data. We propose here optimality criteria for the design of routing networks. For a very general architecture, we derive the number of routing layers and the fanout that minimize the required neural circuitry.

Analysis of cluttered scenes using an elastic matching approach for stereo images.

We present a system for the automatic interpretation of cluttered scenes containing multiple partly occluded objects in front of unknown, complex backgrounds. The system is based on an extended elastic graph matching algorithm that allows the explicit modeling of partial occlusions. Our approach extends an earlier system in two ways.

Maplets for correspondence-based object recognition.

We present a correspondence-based system for visual object recognition with invariance to position, orientation, scale and deformation. The system is intermediate between high- and low-dimensional representations of correspondences. The essence of the approach is based on higher-order links, called here maplets, which are specific to narrow ranges of mapping parameters (position, scale and orientation), which interact cooperatively with each other, and which are assumed to be formed by learning.

Image representation by complex cell responses.

We present an analysis of the representation of images as the magnitudes of their transform with complex-valued Gabor wavelets. Such a representation is a model for complex cells in the early stage of visual processing and of high technical usefulness for image understanding, because it makes the representation insensitive to small local shifts. We show that if the images are band limited and of zero mean, then reconstruction from the magnitudes is unique up to the sign for almost all images.

Rapid processing and unsupervised learning in a model of the cortical macrocolumn.

We study a model of the cortical macrocolumn consisting of a collection of inhibitorily coupled minicolumns. The proposed system overcomes several severe deficits of systems based on single neurons as cerebral functional units, notably limited robustness to damage and unrealistically large computation time. Motivated by neuroanatomical and neurophysiological findings, the utilized dynamics is based on a simple model of a spiking neuron with refractory period, fixed random excitatory interconnection within minicolumns, and instantaneous inhibition within one macrocolumn.

Learning the Gestalt rule of collinearity from object motion.

The Gestalt principle of collinearity (and curvilinearity) is widely regarded as being mediated by the long-range connection structure in primary visual cortex. We review the neurophysiological and psychophysical literature to argue that these connections are developed from visual experience after birth, relying on coherent object motion. We then present a neural network model that learns these connections in an unsupervised Hebbian fashion with input from real camera sequences.

Computer-based recognition of dysmorphic faces.

Genetic syndromes often involve craniofacial malformations. We have investigated whether a computer can recognize disease-specific facial patterns in unrelated individuals. For this, 55 photographs (256 x 256 pixel) of patients with mucopolysaccharidosis type III (n=6), Cornelia de Lange (n=12), fragile X (n=12), Prader-Willi (n=12), and Williams-Beuren (n=13) syndromes were preprocessed by a Gabor wavelet transformation.

The role of complex cells in object recognition.

Primate's primary visual cortex (V1) is dominated by complex cells. This choice of nature seems puzzling, as complex cells are insensitive to spatial phase--information which is generally believed to be essential for perceptual characterization and recognition of images. Modeling complex cells as Gabor wavelet magnitudes, we have mathematically and empirically examined the information content of their responses.

Acquisition of visual shape primitives.

Shape primitives have long been proposed as components for object models in the visual system, and account for a considerable body of behavioral findings. While a large amount of effort has been devoted to the study of detection of these parts in the scenes, no research has been undertaken simulating the acquisition of these representations. We present a model which suggests how the shape primitives may be learned by experience in a self-organized fashion.