A Reddit user benchmarked the locally quantized ISTA-DASLab DeepSeek-V4.1-Flash-1017 model at 2.25bpw against the official API using a local Project-maya v1.1.27 setup. The evaluation utilized a 100-question text-only subset of the Humanities Last Exam benchmark to assess the quality of the aggressive quantization. Specific optimization settings, including `DS4_INDEX_TC` and `MAYA_DS4_PROMPT_SKIP`, were configured to maximize local generation quality.

Read original