LuthorSQL is a 7B non-autoregressive text-to-SQL model designed as a fast System 1 gate for slower LLM agents (System 2), escalating only uncertain queries. It performs a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and executes up to 32 candidate plans, and selects answers via a trained validator. On BIRD's hidden test set, LuthorSQL achieves 68.03% execution accuracy at 0.84 seconds per question on a single GPU, retaining accuracy with 8-bit quantization in 9 GB VRAM. It transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of native SQLite performance. In a pre-registered confidence cascade with Claude Opus 5.5 over 227 questions on post-training-cutoff databases, LuthorSQL handles nearly half locally, reducing API costs by 33.3% without accuracy loss. Contamination analysis reveals Opus reproduces 48-92% of public benchmark gold queries verbatim without schema access, yet tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases.
Read original
huggingface/daily-papers