On quasi-dominant weights and Hilbert series of determinantal varieties.

The coordinate rings of the classical determinantal varieties are each isomorphic to a classical invariant ring by Weyl's fundamental theorems of invariant theory. Since these rings are Cohen-Macaulay, their Hilbert series are rational functions whose numerator polynomials have nonnegative integer coefficients. In the case of general determinantal varieties, as well as in the case of symmetric determinantal varieties, these numerator polynomials were shown to be equal to the Hilbert series of certain finite-dimensional highest weight modules and were given an explicit combinatorial description. The current work extends these results to the alternating determinantal varieties. The proof of these results, in all three cases, relies on the fact that the coordinate rings of the determinantal varieties carry the structure of a Wallach representation. The Hilbert series of the Wallach representation is a rational function whose numerator polynomial is given by the Hilbert series of a finite-dimensional highest weight module, and the Hilbert series of the determinantal variety is equal to the Hilbert series of the Wallach representation. T. J. Enright and J. F. Willenbring introduced the more general class of quasi-dominant weights and showed that if L is a unitarizable highest weight module with quasi-dominant highest weight, then the Hilbert series of L is of the form H_L(t) = R * (H_E(t)) / ((1-t)^D), where R is a rational number, E is a finite-dimensional highest weight module, and D is the Gelfand-Kirillov dimension of L. The set of quasi-dominant weights has an interesting characterization in terms of parabolic category O and Kostant's minimal length coset representatives. We give a new characterization in terms of associated varieties and show that the subset of quasi-dominant weights whose highest weight modules occur in the setting of Howe dual pairs has a nice description in terms of the highest weights of the "Howe dual'' representations. Finally, we give some new results on the number of quasi-dominant reduction points.