The author benchmarked four large language models—DeepSeek‑Coder‑V2‑Instruct, Claude 3.5 Sonnet, GPT‑4o, and Gemini 1.5 Pro—on a suite of 15 low‑level programming tasks that involved complex pointer arithmetic, manual memory management, loop unrolling, and data‑sorting arrays in both C and Rust. All models achieved over a 90 % pass rate for syntactic correctness and successful compilation, but their runtime efficiency varied significantly. In C pointer‑heavy challenges, DeepSeek‑Coder‑V2‑Instruct produced code that executed up to 18 % faster than GPT‑4o by eliminating redundant stack‑to‑heap copies. For Rust generation, Claude 3.5 Sonnet consistently delivered zero‑cost abstractions that satisfied the borrow checker on the first attempt, whereas Gemini 1.5 Pro frequently resorted to unsafe blocks to compile complex ownership patterns. Although Claude slightly outperformed DeepSeek in code readability and commenting, it did not match DeepSeek’s micro‑optimization capabilities. The author suggests extending the benchmark to evaluate model‑generated optimizations for specific instruction‑set architectures such as ARM versus x86.

Read original