The paper introduces SkewAdam, an optimizer designed to reduce the memory overhead of optimizer states during Mixture-of-Experts (MoE) training. By recognizing that dense backbones, experts, and routers exhibit different gradient statistics, SkewAdam employs tiered state allocation rather than a uniform approach. This method aims to optimize memory efficiency, addressing the fact that optimizer states often constitute the largest portion of the memory budget.