The developer added OpenVINO support to the Laya framework, enabling CPU‑based inference. This achieves an average latency of 40 ms per question, which is 3.4× faster than the PyTorch implementation. The code is available in the rupeshs/laya‑openvino repository, with a Flappy Bird demo at rupeshs/flappy‑laya‑openvino‑cpu.
Read original
reddit/r/LocalLLaMA