Invariant theory and orientational phase transitions.

The Landau theory of phase transitions has been productively applied to phase transitions that involve rotational symmetry breaking, such as the transition from an isotropic fluid to a nematic liquid crystal. It even can be applied to the orientational symmetry breaking of simple atomic or molecular clusters that are not true phase transitions. In this paper, we address fundamental problems that arise with the Landau theory when it is applied to rotational symmetry breaking transitions of more complex particle clusters that involve order parameters characterized by larger values of the l index of the dominant spherical harmonic that describes the broken symmetry state. The problems are twofold. First, one may encounter a thermodynamic instability of the expected ground state with respect to states with lower symmetry. A second problem concerns the proliferation of quartic invariants that may or may not be physical. We show that the combination of a geometrical method based on the analysis of the space of invariants, developed by Kim to study symmetry breaking of the Higgs potential, with modern visualization tools provides a resolution to these problems. The approach is applied to the outcome of numerical simulations of particle ordering on a spherical surface and to the ordering of protein shells.