OK guys, let's be honest 1 minute about local LLM
Article automatically generated from technical news.
Ngl, I keep seeing posts from people running local LLMs. Most of the time the setup is a 3080, 3090, 4090, 5090, sometimes a whole mini lab. Cool. I get it. It's fun to test every open source model that drops. I do that too sometimes. But real talk. Are there people here who actually work with local LLMs daily? Like for real tasks, not just benchmarks and "look, it runs"? What do you actually use them for? Because every time I try to make it practical, I hit the same wall. Tokens
Fonte originale