Progress in Mathematics Volume 123 Series Editors J. Oesterle A. Weinstein Lie Theory and Geometry In Honor of Bertram Kostant Jean-Luc Brylinski Ranee Brylinski Victor Guillemin Victor Kac Editors Springer Science+Business Media, LLC Jean-Luc Brylinski Ranee Brylinski Department of Mathematics Department of Mathematics Penn State University Penn State University University Park, PA 16802 University Park, PA 16802 Victor Guillemin Victor Kac Department of Mathematics Department of Mathematics MIT MIT Cambridge, MA 02139 Cambridge, MA 02139 Library of Congress Cataloging In-Publication Data Lie theory and geometry : in honor of Bertram Kostant / Jean-Luc Brylinski... [et al.], editors. p. cm. - (Progress in mathematics ; v. 123) Invited papers, some originated at a symposium held at MIT in May 1993. Includes bibliographical references. ISBN 978-1-4612-6685-3 ISBN 978-1-4612-0261-5 (eBook) DOI 10.1007/978-1-4612-0261-5 1. Lie groups. 2. Geometry. I. Kostant, Bertram. II. Brylinski, J.-L. (Jean-Luc) III. Series: Progress in mathematics : vol. 123. QA387.L54 1994 94-32297 512'55~dc20 CIP Printed on acid-free paper © Springer Science+Business Media New York 1994 Originally published by Birkhäuser Boston in 1994 Softcover reprint of the hardcover 1st edition 1994 Copyright is not claimed for works of U.S. Government employees. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopy• ing, recording, or otherwise, without prior permission of the copyright owner. Permission to photocopy for internal or personal use of specific clients is granted by Springer Science+Business Media, LLC, for libraries and other users registered with the Copyright Clearance Center (CCC), provided that the base fee of $6.00 per copy, plus $0.20 per page is paid directly to CCC, 21 Congress Street, Salem, MA 01970, U.S.A. Special requests should be addressed directly to Springer Science+Business Media, LLC. ISBN 978-1-4612-6685-3 Text prepared by TeXniques, Inc. from disks. 987654321 Bertram Kostant Table of Contents Preface . ix Normality of Some Nilpotent Varieties and Cohomology of Line Bundles on the Cotangent Bundle of the Flag Variety Bram Broer 1 Holomorphic Quantization and Unitary Representations of the Teichmiiller Group Jean-Luc Brylinski and Dennis McLaughlin ........... 21 Differential Operators on Conical Lagrangian Manifolds Ranee Brylinski and Bertram Kostant .............. 65 Groups and the Buckyball Fan R. K. Chung, Bertram Kostant, and Shlomo Sternberg . .... 97 Spinor and Oscillator Representations of Quantum Groups Jintai Ding and Igor B. Frenkel. 127 Familles coherentes sur les groupes de Lie semi-simples et restriction aux sous-groupes compacts maximaux Michel Duflo and Michele Vergne . 167 The Differential Geometry of Fedosov's Quantization Claudio Emmrich and Alan Weinstein . ... 217 Closedness of Star Products and Cohomologies Moshe Flato and Daniel Sternheimer .... 241 The Algebra of Chern-Simons Classes, the Poisson Bracket on it and the Action of the Gauge Group Israel M. Gelfand and Mikhail M. Smirnov . .. 261 A Distinguished Family of Unitary Representations for the Exceptional Groups of Real Rank = 4 Benedict H. Gross and Nolan R. Wallach .... 289 Reduced Phase Spaces and Riemann-Roch Victor Guillemin . 305 viii Table of Contents The Invariants of Degree up to 6 of all n-ary m-ics Roger Howe ................ 335 Tensor Products of Modules for a Vertex Operator Algebra and Vertex Tensor Categories Yi-Zhi Huang and James Lepowsky ..... 349 Enveloping Algebras: Problems Old and New Anthony Joseph . ............ 385 Integrable Highest Weight Modules Over Affine Superalgebras and Number Theory Victor G. Kac and Minoru Wakimoto . 415 Quasi-Equivariant V-Modules, Equivariant Derived Category, and Representations of Reductive Lie Groups Masaki Kashiwara and Wilfried Schmid ...... 457 Meromorphic Monoidal Structures David Kazhdan ........ 489 The Nil Hecke Ring and Singularity of Schubert Varieties Shrawan Kumar 497 Some Classical and Quantum Algebras Bong H. Lian and Gregg J. Zuckerman 509 Total Positivity in Reductive Groups George Lusztig . 531 The Spectrum of Certain Invariant Differential Operators Associated to a Hermitian Symmetric Space Siddhartha Sahi. 569 Compact Subvarieties in Flag Domains Joseph A. Wolf . ......... 577 The Work of Bertram Kostant Over a period of more than four decades Kostant has produced some extremely creative, important and influential mathematics. It continues to be very striking to all of us that, among his many papers, virtually all are pioneering with deep consequences, many an article spawning a whole field of activity. While there have been many papers that either further develop or generalize papers by him, few have improved on his treatment of any topic. Most of Kostant's work is related to Lie theory and its many facets, including algebraic groups and invariant theory, the geome­ try of homogeneous spaces, representation theory, geometric quantization and symplectic geometry, Lie algebra cohomology, Hamiltonian mechanics, modular forms, etc... In fact, one striking consequence of Kostant's body of work is that Lie theory (and symmetry in general) now occupies a larger role in mathematics than it did before him. Kostant's depth of range ex­ tends to major papers in algebra and mathematical physics, and includes a number of unpublished papers which have also been very influential. Much of Kostant's mathematics demonstrates the value and the im­ portance of achieving the necessary level of generality at the same time as new ground is broken. By the same token, however, his work illustrates his own belief that there are certain objects and structures in mathematics which are so beautiful and so fundamental that they justify many years of effort to unearth their hidden treasures. The present volume is a tribute to Bert Kostant. The 22 high level articles assembled here reflect the diversity of his influence on many fields. While the excellence of the articles will speak for themselves, we will at­ tempt to give an overview of Kostant's work to date (early July, 1994). We have chosen to divide the exposition according to the decades of his scien­ tific activity, as this seems to lead to a more lively picture of the ensemble of his work. We have attempted to at least illustrate the variety of the papers, their origin, their influence on mathematics (and also on physics); we have not tried to discuss the works of other mathematicians except to the extent that they are directly related to or have influenced Kostant's work and his thinking. No doubt much has been omitted and many con­ nections have escaped our notice. We hope that this overview can serve as a guide, as experience shows again and again that so much is gained from reading the original source of a discovery, and Kostant's papers are very well-written. There was a Symposium in honor of Kostant at MIT in May, 1993. Some (but hardly all) of the papers in this volume grew out of talks x Preface given there. The Symposium, entitled "Lie Theory, Algebra and Geomet­ ric Quantization", seems to be a fitting descriptive synthesis of Kostant's works. This summary was prepared by the Editors. We thank David Vogan for giving us his account of Kostant's work in representation theory up to the mid 1970s. Any inaccuracies are of course the responsibility of the Editors. 50s Some of Kostant's first papers ([1] [4] [5]) are devoted to a study of the holonomy groups of homogeneous spaces. Kostant described the Lie al­ gebra of the holonomy group in terms of certain operators Ax associated with Killing fields X. In [4] and [5] the holonomy group is described for a large class of homogeneous spaces. In [10] Kostant gives a characteri­ zation of those affine connections which admit locally a transitive group of connection preserving transformations. These results are reproduced in the treatise of Kobayashi and Nomizu. In [3] it is proved that for any Rie­ mannian metric on the sphere sn, the holonomy group is always the full rotation group. This basic result is derived from a representation-theoretic fact proved in [10], namely, that if a subgroup G of SO(n) (for n ~ 5) has no invariants in I\i jRn for all 0 < i < n, then G = SO(n). Kostant gave in [2] a complete classification of real Cartan subalgebras of a real simple Lie algebra. The classification is done in terms of suitable families of orthogonal roots. In the second part of [2] there are complete tables for all the Cartan subalgebras. This second part was widely circu­ lated, but remained unpublished as the PNAS complained that the tables were too complicated! The first (and independent) publication of the lists was in a paper by Sugiura (the lists of Kostant and Sugiura coincide). In [8] Kostant gave his famous formula for the multiplicity of a weight J.L in a finite dimensional representation V,x of a simple Lie algebra g. The mul­ tiplicity is the alternating sum over w in the Weyl group of the expression S;P(w· (,X + p) - (J.L + p», where s;P denotes the Kostant partition function, which counts the number of expressions of a weight as a sum of posi­ tive roots. The Kostant multiplicity formula is a precursor to the theory of Verma modules and to the Bernstein-Gelfand-Gelfand resolution of a finite-dimensional representation. It served as a model for many subse­ quent character formulas (for instance, Blattner's formula). The paper [8] also contains some other curious identities, which deserve to be studied (see the review of [8] by Freudenthal). There is much work of Kostant on generalizations of [8] which has remained unpublished - notably he has a formula for the restriction of a representation to an arbitrary reductive Preface xi subgroup, and another result deals with a subalgebra which is the fixed point algebra of some involution.