Infermeld is an experimental open-source Linux kit that allows a single GGUF model to run across both AMD and NVIDIA GPUs using llama.cpp. The v0.1.0 release seeks users with mixed GPU configurations to test the setup and identify issues. It aims to enable unified inference on existing multi-vendor hardware without buying new cards.
Read original
reddit/r/LocalLLaMA