Mathematical aspects of molecular replacement. V. Isolating feasible regions in motion spaces.

This paper mathematically characterizes the tiny feasible regions within the vast 6D rotation-translation space in a full molecular replacement (MR) search. The capability to a priori isolate such regions is potentially important for enhancing robustness and efficiency in computational phasing in macromolecular crystallography (MX). The previous four papers in this series have concentrated on the properties of the full configuration space of rigid bodies that move relative to each other with crystallographic symmetry constraints. In particular, it was shown that the configuration space of interest in this problem is the right-coset space Γ\G, where Γ is the space group of the chiral macromolecular crystal and G is the group of rigid-body motions, and that fundamental domains F can be realized in many ways that have interesting algebraic and geometric properties. The cost function in MR methods can be viewed as a function on these fundamental domains. This, the fifth and final paper in this series, articulates the constraints that bodies packed with crystallographic symmetry must obey. It is shown that these constraints define a thin feasible set inside a motion space and that they fall into two categories: (i) the bodies must not interpenetrate, thereby excluding so-called `collision zones' from consideration in MR searches; (ii) the bodies must be in contact with a sufficient number of neighbors so as to form a rigid network leading to a physically realizable crystal. In this paper, these constraints are applied using ellipsoidal proxies for proteins to bound the feasible regions. It is shown that the volume of these feasible regions is small relative to the total volume of the motion space, which justifies the use of ellipsoids as proxies for complex proteins in MR searches, and this is demonstrated with P1 (the simplest space group) and with P222 (the most common space group in MX).