To mitigate GPU fragmentation caused by maintaining multiple legacy models, researchers developed a consolidated self-hosted LLM designed to handle diverse corporate workloads. By analyzing production error patterns across instruction following, function-calling, and internal task distribution, the team identified specific failure modes: semantic collapse, over-calling, and verbosity hacking. Instead of simultaneous multi-objective optimization, which caused reward interference, the authors trained specialized experts for each axis using Group Relative Policy Optimization (GRPO). These experts were subsequently integrated into a single model via a two-stage Spherical Linear Interpolation (SLERP) merging process. The resulting model successfully absorbed 50% of platform traffic, processing approximately 116 million monthly requests. In non-reasoning modes, the optimized model outperformed a baseline with ~7x more parameters across several metrics: it achieved a 69.6 to 65.8 score on the in-house Arena, an instruction following score of 0.83 (up from 0.85 in specific comparative contexts), and a function-calling score of 0.77 (up from 0.79). This approach allowed the enterprise to consolidate traffic from over 200 internal applications onto a single architecture, significantly reducing serving costs while maintaining high performance on stratified offline benchmarks evaluated through deterministic verifiers and calibrated LLM judges.

Read original