Researchers have introduced Comparison-based Preference Optimization (ComPO), a novel zeroth-order paradigm designed to align large language models (LLMs) with human preferences. Unlike traditional direct alignment methods that rely on differentiable preference losses—which can suffer from likelihood displacement when preference margins are small—ComPO utilizes comparison oracles to extract directional information from preference pairs. The authors provide theoretical foundations for both offline and online schemes. The offline scheme is proven to converge under conditions of smoothness, gradient sparsity, and compatibility between the oracle and a latent objective. To prevent distribution shift, the online ComPO variant incorporates unlabeled policy generations to maintain reverse-KL control relative to a reference policy. Furthermore, the study establishes performance guarantees for a constrained scheme based on local coverage and in-distribution pairwise reward accuracy. Empirical evaluations conducted on several state-of-the-art models, including Mistral, Llama, Gemma-2, Qwen3, and Gemma-3, demonstrate that ComPO outperforms existing direct alignment methods. Notably, the method shows significant improvements in length-controlled win rates, and pair-level diagnostic analysis confirms its effectiveness in mitigating the likelihood displacement issues common in standard preference fine-tuning approaches.