Zyphra introduces Zyphra Hierarchical Attention (ZHA), a sparse attention mechanism that replaces the flat indexer scan with a tree-structured search over the KV cache, targeting the O(N) indexing bottleneck that consumes 60–90% of layer time near 1M tokens. ZHA compresses contiguous key chunks into summary keys using learned reference queries (16 per configuration) that estimate log attention mass via tangent-plane approximations; these summaries are hierarchically aggregated across levels (e.g., chunk size 4, 6 levels covering 4,096 tokens per top node). At decode, a query routes top-p (p=0.95) through the tree, expanding only branches covering the target attention mass fraction, yielding adaptive sparsity. Applied to frozen Qwen3.6-35B-A3B with only 81,920 trainable parameters (0.00024%), ZHA trains via KL divergence matching dense attention mass per chunk over 3.98B tokens at 32K context. A deep tree (chunk 4, 6 levels) achieves 63.4% on LongBench v2 versus 58.0% dense while reading ~1.1% of keys, and 93.3% on RULER-13 (256K) versus 96.0% dense; a wide tree (chunk 256, 1 level) matches dense at 96.2% RULER-13. Both surpass MiniMax Sparse Attention (MSA), which drops to 77.1% on RULER-13 despite 40B-token training. On Nemotron-3.5-Lightning-30B-A3B (NoPE), deep-tree RULER-13 falls 25.3 points below dense. Modeled H100 BF16 roofline projections (no kernels released) show ZHA beating dense attention near 48K tokens and MSA near 480K tokens, reaching 373–417x over dense and 52–58x over MSA at 1B tokens; at 256K, Pareto settings yield 1.93–2.28x speedups at 92.7–96.4% RULER-13 accuracy.