Speculative cache warming: warms your cache while you type your prompt, save 10-20s of wait time

Article automatically generated from technical news.

https://preview.redd.it/0g9l1pvqsdch1.png?width=603&format=png&auto=webp&s=33b554fa2e8344205dc586fb4080bb4e472c8abb Hello, I'm continuously working on OpenFox (MIT-licensed - no business model whatsoever), which is a harness dedicated to local AI, mostly for coding but well you know, this can do anything. I'm using it every day with my 2x Spark cluster, mostly with DS4 Flash these days. I noticed a small opportunity for improvement, nothing revolutionary