GeoPair introduces a training‑free compression framework that targets cross‑layer redundancies in transformer models by jointly optimizing weight pairings and learning a shared dictionary factorization. Rather than forcing adjacent layers to adopt a common basis or merging activation statistics heuristically, the method first identifies structurally compatible projection subspaces between layers. It then learns a shared representation that captures the common structure while preserving each layer’s individual calibration geometry, ensuring that the functional behavior of each layer remains intact. This geometry‑preserving factorization is combined with structured sparsity patterns to produce compact weight decompositions that can be applied without any further training. The optimization proceeds sequentially across layers, converging to a set of cross‑layer weight pairs and dictionary atoms that minimize reconstruction error under the sparsity constraints. Experiments reported in the paper show that GeoPair consistently outperforms baseline approaches that perform independent structured weight decompositions or rely on heuristic pairwise factorizations, achieving state‑of‑the‑art compression‑accuracy trade‑offs across a variety of transformer architectures, model scales, and modalities (e.g., language, vision, and multimodal models). By replacing heuristic grouping with an optimization‑driven pipeline, the work provides a theoretically grounded, scalable solution for post‑training transformer compression.