Ollaya is a local server that runs open‑source decision‑model weights to answer typed questions about arbitrary text or JSON in a single forward pass, delivering calibrated probabilities in milliseconds. It ships several models: laya (fastest, 322 M parameters), decider (most accurate, 0.75 B parameters, scores 0.591 on typed‑decision benchmarks), von (handles up to 8 192‑token inputs), qwen3guard (safety classifier in 119 languages), and others such as nli, gliclass, kev and decision. On an RTX 4090 GPU, a five‑question query to laya completes in roughly 8‑10 ms per question, while the same workload via the TypeSafe‑compatible Jev API takes 236‑276 ms end‑to‑end; CPU execution is also supported. Ollaya exposes the TypeSafe endpoints /v1/systemone and /v1/models, so the official TypeSafe Python SDK 0.7.1 works unchanged when pointed at http://localhost:11435. Models are downloaded as verified Hugging Face checkpoints, run with ONNX Runtime, and can be recalibrated with a user‑supplied Modelfile. The project provides desktop apps for macOS, Windows and Linux, a CLI, and Docker images, all under an Apache‑2.0 license, with GPU acceleration available on NVIDIA CUDA 13‑capable hardware and Apple GPUs via MLX for laya and nli.

Read original