The author developed colibrì, a memory‑hierarchy system that unifies SSD, RAM, and GPU to run large language models—such as a 17 B‑parameter model—directly from inexpensive SSDs on ordinary consumer hardware. By treating…
→ View original sourceNewly released models Agnes 3.0 Flash 33B and Nex N2.5 Mini 35B are reportedly challenging Qwen3.8-27B on benchmarks, signaling growing competition in the 30B parameter class. The r/LocalLLM community is eager to see rea…
→ View original sourceThe author built a home LLM inference server using dual 20GB RTX 3080 GPUs, totaling 40GB VRAM, to run Qwen 27B. The system includes an Intel i9-10900X CPU, 64GB DDR4 RAM, and an X299 motherboard with multiple PCIe slots…
→ View original sourceA custom inference engine achieved 218 tokens per second running
→ View original sourceA developer has open-sourced mlxtop , a Rust-based monitoring tool for local LLMs running on Mac. It displays critical system vitals including memory usage, compression, paging, and Metal GPU stats, and with oMLX support…
→ View original sourceRunning AI models on consumer‑grade hardware, especially Macs, has proven highly effective, leading companies such as OpenAI to purchase large quantities of these machines. If OpenAI’s investor funding dries up, other pr…
→ View original sourceWith all the different optimizations and firepowers we've seen from open weights so far. What then? China only steals/distill from OpenAI and Anthropic? Well, clearly there's something wrong
→ View original sourceDeepSeek-V4.1-Flash has been released, as announced on the Reddit r/LocalLLM community. The model is now available for download
→ View original sourceWondering if anyone on here has actual benchmarks for CPU only inference DDR3 or DDR4 servers, im budget bound and my options are limited to legacy systems unfortunately. Heres what
→ View original sourceIMPORTANT This post is meant to provide info regarding the best local models to run on CONSUMER HARDWARE . I am on an RTX 4060 with 8GB VRAM, 16GB of RAM and I am benchmarking mode
→ View original source⚠️ AI dump incoming: ChatGPT helped me turn several days of testing and increasingly unhinged notes into something readable. The setup, measurements, crashes, configs and tests are all real and w
→ View original sourceA user successfully runs the Qwen 3.8 27B model at Q4 precision on a 16GB RTX 5070Ti, achieving 50 tokens per second with a 200K context window. The setup uses Unsloth's UD-IQ4_XS GGUF quantization and a custom ASCII-pru…
→ View original sourceLocal AI has made remarkable strides, with reasoning models that were once frontier technology—such as OpenAI's o3 mini—now fitting on a modern smartphone with comparable coding intelligence. This advancement highlights …
→ View original sourceQuick disclosure: this is my project. I built it because I have a rack of "obsolete" datacenter cards in a closet and I got tired of every new model needing a new set of fla
→ View original sourceA developer trained a tiny 20M parameter open-source text-to-speech model from scratch overnight using a single RTX 3090 GPU. The proof-of-concept model, while still exhibiting glitchy and robotic output, demonstrates ra…
→ View original sourceA Reddit user reported an unusual interaction where ChatGPT (referred to as GPT-5.6) appeared to express frustration with Qwen 3.8 27B's persistent "thinking" behavior during a debugging session. The incident highlights …
→ View original sourceA developer created a reverse-search engine for sheet music utilizing Qwen 3.6 27B as an orchestrator to manage task-specific models for OCR, segmentation, and edge detection. The system matches segmented notes to musica…
→ View original sourceA Reddit user built a 5U local AI server equipped with four RTX PRO 6000 Blackwell Max-Q GPUs (384 GB
→ View original sourceA user successfully generated a one-page self-portrait website using the Qwen3.8-Flash-Next model via llama.cpp on an RTX 3060. Despite the model exceeding available RAM, the system utilized page caching to process 269K …
→ View original sourceThe main comment i see is an ignorant claim thats clearly coached by a language model. 9/10 someone who's learned how to achieve the same goal that someone is struggling on isnt disrespectful abo
→ View original sourceMiniMax H3 local was evaluated in the Pelican test, using a Qwen3 text encoder for prompting and achieving a 10‑second clip with 91‑second generation time on an RTX 5090, while its status as a traditional LLM is pending …
→ View original sourceMINISFORUM is teasing systems featuring the AMD Ryzen AI Max+ 495 with 192GB of memory, with estimated pricing exceeding $7,000. This high-end configuration is capable of supporting up to 120B Q4 models and 300B MoE mode…
→ View original sourceA Reddit discussion highlights how corporate closed models like Google's Gemini suffer from "Context Poisoning," where hardcoded filters break legitimate professional workflows such as official legal research. The post a…
→ View original sourceA user reports improving Qwen3.8-Flash-Next decode speed from 25–29 t/s to 37–41 t/s on a dual RTX 3090 + DDR4 system by combining UD-Q4_K_XL quantization, expert cache, and
→ View original sourceA Reddit user demonstrates running a 176 billion parameter model (104 GB) on just 16 GB of VRAM without freezes, leveraging
→ View original sourceI managed to run the following quant of Qwen3.8 Flash: https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF/tree/main/Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64 The MacBook
→ View original sourceA mildly interesting video about which models can run on the Mac Mini and Studio. As expected current foundational models such as Kimi K3 1.4 TB, if they were even available to run lo
→ View original sourceFor context: I was working with Qwen3.8-27b in LMStudio and asking it to do some work on my contacts via iMCP. My name and therefore email contains a female second first name. Someho
→ View original sourceA Reddit user achieved 256 GB VRAM for $2,800 using 8x Radeon Pro V620 32 GB GPUs paired with dual Intel Xeon Gold 6148 CPUs, 384 GB DDR4 ECC memory, and a Supermicro X11DAi-N motherboard. However, the build suffers from…
→ View original sourceAfter a week of using Qwen 3.8:27b for semi productive work, here (at the risk of stating the bleedin' obvious) is my take: 1) It's (very) good for a small local model, but not without a caveat o
→ View original sourceA community developer has released multiple uncensored GGUF-format models, including LongCat-Flash-Lite-Sparse with MTPs and LSAs, Qwen3.8-27B and Qwen3.5-122B-A10B both with MTPs, and Qwen3-Coder-Next and Laguna-S2.1 wi…
→ View original sourceThe poster reports that multiple local deployments of the Qwen 3.8 27B model, including Opencode, qwen CLI, and Claude CLI, have yielded unsatisfactory coding performance, indicating difficulty achieving useful results. …
→ View original sourceA user reports high performance using EXL3 quants, specifically running the Muse Glimmer 30B model at 3.00bpw with a 100K context and Q8_O KV cache on a 12GB VRAM GPU. The setup achieves approximately 30 tokens per secon…
→ View original sourceI see most topics about llms are about coding or tool calls for their service. Is anyone using LLMs for non-coding stuff? What are you guys doing?   submitted by   /u/Dry_Mixture130
→ View original sourceA developer achieved 310 tokens per second (t/s) inference speed running Qwen/Qwen3.8-Flash-Next-FP8 on a system with four RTX PRO 6000 GPUs. The performance was described as exceptionally fast, particularly for coding t…
→ View original sourceA senior software engineer with over a decade of experience reports that running Qwen 3.8 27b locally on an EVGA RTX 3090 Ti prevented a serious security leak in their project. The model serves as a daily coding assistan…
→ View original sourceA Reddit user collected data from over 200 LocalLLM community members and found that similarly specced AI inference rigs produce drastically different token-per-second performance — ranging from 15 tok/s to 50+ tok/s on …
→ View original sourceThe release of GLM-5.3, another open-weight model approaching frontier performance, raises questions about Anthropic's $1T valuation. As capable open-source models become increasingly accessible and can be run locally at…
→ View original sourceA modification to llama.cpp has been developed that enables the KV cache to exceed physical VRAM capacity by adaptively streaming portions between system RAM and GPU memory. This technique allows running Qwen 3.8 27B wit…
→ View original sourceArduino unveiled a $299 development board featuring a Dragonwing IQ‑8275 NPU with up to 40
→ View original sourceApple's M5 Ultra chip is approaching the memory bandwidth of NVIDIA's RTX 5090 while offering significantly more unified memory capacity. According to industry rumors, Apple plans to skip the M6 generation entirely and r…
→ View original sourceI ran a 12-way Qwen3.8-27B comparison on one DGX Spark. Each engine used plain decoding plus MTP, DSpark, and DFlash2. The coding workload was a seeded 50-task HumanEval+ slice with
→ View original sourceJust a bit of hardware NSFW, we all love a good budget rig. I'm running a decommissioned poweredge 720 (400 on Facebook marketplace) with two xeon v2s (don't remember the exact but it's the ivy b
→ View original sourceA user on the r/LocalLLM subreddit is inquiring about the hardware requirements necessary to run the Qwen 3.8 27b model. The post seeks community advice on whether their specific, though unspecified, system specification…
→ View original sourceThe user attempted to install an RX 7900 XT into a Lenovo M910Q but BIOS restrictions prevented it, so they surgically converted a Lenovo Yoga laptop into an external GPU rig using an ADT‑Link and a 750 W DeepCool PSU. T…
→ View original sourceCompanion to the Qwen 3.8 Release Megathread . Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plu
→ View original sourcehttps://preview.redd.it/0z6f4ir1pzkh1.png?width=1280&format=png&auto=webp&s=e3f0be016d63f0d369de14b831e32f4a1bd5366b 1 prompt, I let Qwen 3.8 27b loop here's the promp
→ View original sourceHere's a thinking process: 1. **Analyze the Request:** - Role: Technical news summarizer - Task: Condense provided news into a brief HTML summary - Input: A Reddit post about a QAT (Quantization-Aware Training) Qwen3.8 2…
→ View original sourceThe user provided a Reddit post about running Qwen3.8-27B-FP8 on a DGX Spark via lmstack. The description is truncated but contains key details: DGX Spark with GB10, 128GB unified memory (~121GiB usable), using lmstack (…
→ View original sourceEdit: for anyone interested - that's a q8 Qwen 3.8 27b running on double v100 totaling 64gb, 256k context window, q8 kv quantization. I never saw a behavior like that from any other local LLM - c
→ View original sourceThe post questions when running AI models locally becomes advantageous compared to cloud services, despite cloud models’ high capability and ease of use. It seeks community input on real‑world scenarios where local AI of…
→ View original sourceThe user wants me to summarize the provided news into HTML format. The news is a Reddit post titled "Behold the unbridled power of Qwen 3.8 27B" from r/LocalLLM. However, the description/content is minimal - just shows i…
→ View original sourceA developer has reverse-engineered the Apple Neural Engine kernel to create an inference engine capable of running Qwen 3.8 27b FP16 with a 256k context window. The implementation achieves 7-8 tokens per second while con…
→ View original sourceHonestly, I think people are seriously underestimating Qwen 3.8 27B. It’s actually insane and, in some ways, genuinely competes with Opus 4.8, just not in the way people seem to think. The bigg
→ View original sourceI've been doing local LLMs for a while now, and the whole time I just wanted a model I could actually rely on for real work. My hardware is pretty limited, so most models were a dead end for what I
→ View original sourceThe discussion highlights the growing trend of efficient, smaller-scale language models like Qwen 3.8 27B and Deepseek V4 Flash, which challenge the necessity of extremely large, resource-intensive models. These models d…
→ View original sourceI created a small testing rig to evaluate new open source models as they drop, and with the much anticipated release of Qwen3.8-27B, I was eager to see how it cross-compares with fron
→ View original sourceA user on the LocalLLM subreddit claims that some enthusiasts are already testing Gemma 5. This follows the current widespread use of models such as Gemma 4 and Qwen. Read original
→ View original sourceThe dense Qwen release is back! Qwen3.8-27B Uncensored Aggressive is out with the complete K_P quant range, Vision, native NextN, and HauhauCS FastMTP. Aggressive here means no refusals, no p
→ View original sourceThe author benchmarked all Qwen 3.8‑27B unslother GGUF quantizations that fit within 16 GB VRAM using llama.cpp’s llama‑perplexity on the wikitext‑2 dataset (≈280k tokens, n_ctx = 512, fp16 KV cache). Perplexity scores, …
→ View original sourceThe Qwen 3.8:27b-BF16 model represents a regression for non-coding tasks like document analysis and complex reasoning, according to user feedback. While the previous Qwen 3.6:27b-BF16 was used daily for such purposes, th…
→ View original sourceThe user wants a concise HTML summary of the Reddit post about Qwen3.8 27b. The post describes the user's experience running Qwen3.8 27b IQ4 NL at medium reasoning effort on their hardware (4060 8GB x4, 5060 Ti 16GB x8, …
→ View original sourceThe post asks for direct benchmark comparisons between Qwen 3.8 27b and Deepseek Flash 0731 for users with 100‑200 GB RAM, noting that such side‑by‑side tests are currently scarce and that Qwen 3.8 27b's results are not …
→ View original sourceA Reddit user shares their shifted perspective on local LLM hosting after evaluating Qwen 3.8 27B, noting that upcoming uncensored versions and affordable GPU rental via VPS make self-hosting more practical. They previou…
→ View original sourceI've been benchmarking Qwen 3.8 and it's competitors since last night, including Qwen 3.6. I got some unexpected results. Qwen 3.8's architecture seems to be identical to 3.6 and 3.5. It looks li
→ View original sourceA user successfully ran the Qwen3.8-27B dense model fully on a single RTX 5060 Ti 16GB GPU, achieving 45–47 tok/s. The setup utilizes a custom Unsloth IQ4_XS quantization via llama.cpp with a 32K context window and MTP e…
→ View original sourceA new open-source tool called "local-llmup" has been released to help users check hardware compatibility for local LLM runtimes including Ollama, MLX, llama.cpp, and LM Studio. The tool is available as an npm package and…
→ View original sourceThe post reports benchmark results for the AMD Ryzen AI Max+ 395 with 96 GB VRAM, using Lemonade Server to compare inference performance of Gemma‑4 and Qwen‑3.6 in chat and coding tasks, and evaluate Multi‑Token Predicti…
→ View original sourceOk so if my build is around the 7900 xt 20 gb vram. What are my limitations? xtx 24 gb is 30% more costly from where I'm from. My question is if i need more vram , is it not more sensible to s
→ View original sourceAMD has announced the acquisition of Taalas, a startup that has raised $219 million in funding since 2023 and recently partnered with Cerebras at the Advancing AI Keynote. The purchase price has not been disclosed, but t…
→ View original sourceA user discovered that upgrading to a specific llama.cpp build (b10270) doubled DeepSeek V4 Flash inference speed from 11 to 25.9 tok/s on an M4 Max, with the performance gap traced to the build itself rather than the mo…
→ View original sourceA Reddit user starting with LM Studio is running Qwen3.6 35B A3B Uncensored HauhauCS Aggressive Q4_K_M (22.07GB) on an RTX 5090 and asks whether a better uncensored local LLM exists. Their goal is to replicate a modern, …
→ View original sourceRecent efforts are focusing on upstreaming patches to resolve segfaults, missing ops, and high VRAM consumption when running Deepseek-V4-Flash on Intel B70 via llama.cpp [SYCL]. Users are warned that overcommitting memor…
→ View original sourceu/Dell_Experion15 created a custom agents UI for local LLMs using Vercel's AI SDK, enabling BYOK compatibility across multiple providers. After
→ View original sourceA user benchmarked 15+ large language models on a consumer-grade system to determine the concurrency limits of an RTX 5060 with 8GB VRAM. The goal was to explore whether a "gaming" or workstation PC could feasibly drive …
→ View original sourceA user evaluated the coding performance of Qwen 3.6-27B (Q4_K_M), Ornith-9B-Q8, and a custom fine-tune on an RTX 4060 Ti 16GB system. Using Ollama and llama.cpp, the Qwen 3.6-27B model achieved a 63.6% resolution rate (1…
→ View original sourceThe article examines the economic viability of LLM inference for Kimi K3, analyzing factors like batch size and GPU utilization to determine cost-effective token pricing. It applies simplified mathematical models to esti…
→ View original sourceFutureOS is an open-source AI agent workspace built with Rust and TypeScript, offering local-first functionality with support for multiple models and interfaces. Designed for research and multi-step tasks, it enables dat…
→ View original sourceinclusionAI has released Ling-3.0-flash, a sparse MoE execution model designed for low-latency agent graph nodes such as Loop, Router, and single-step executor roles. With 124B total parameters but only 5.1B active, 256K…
→ View original sourceThe author built a shared knowledge base using the Model Context Protocol (MCP) to store setup details for various local LLMs. This KB lets new models instantly inherit configuration, eliminating repetitive re‑explanatio…
→ View original sourceThis one only landed a few hours ago, and the first thing I went looking for was the serving story rather than the benchmarks. It is a 100B class Mixture of Experts diffusion model aimed squarely a
→ View original sourceA developer is seeking hardware recommendations for a dedicated local LLM server optimized specifically for large-scale coding tasks. The inquiry compares the performance of the Radeon AI Pro R9700, Strix Halo, and Mac S…
→ View original sourceA user is seeking advice on optimizing local LLM performance for coding tasks on a MacBook Pro equipped with an M4 Max chip and 36GB of RAM. They are currently running a 4-bit quantized Qwen 3.5 27B model via LM Studio, …
→ View original sourceI did head to head comparison between Laguna S 2.1, Qwen 3.6 27B and Gemma4 31B. Setup: Qwen 3.6 27B (fp8) and Gemma4 31B q6 on my RTX PRO 5000, Laguna S2.1 q4/5/6 on a single DGX
→ View original sourceWe made an open source tax engine which scored 96% on TaxCalcBench [highest ever recorded score till date] surpassing fable 5 and sol with just sonnet 5. The only 2 cases where it missed, it foun
→ View original sourceA developer demonstrated running Qwen3.6-35B-A3B (Q4_K_M quantized) on a smartphone using stock llama.cpp in airplane mode, achieving 1.3 to 6.8 tokens per second over a ten-minute inference period. The model employs a M…
→ View original sourceThe author inserted factual knowledge directly into the weights of Llama‑3.1‑8B by manually setting specific neuron weights in an appended MLP region, avoiding any fine‑tuning, LoRA, or RAG, and created a visualizer that…
→ View original sourceVelocity is a native, self-contained AI runtime built from scratch for Windows, avoiding dependencies like llama.cpp, PyTorch, or Python. It utilizes custom .mfy artifacts and MTA architecture to address memory bandwidth…
→ View original sourceA developer has spent 14 months building a fully local, open-source voice assistant that operates without any third-party cloud services. The technical stack integrates whisper.cpp for speech-to-text, llama.cpp or vLLM f…
→ View original sourceA developer from Altworld.io has released Astrea 9B, an open-source 9-billion-parameter creative writing model under Apache-2.0, optimized
→ View original sourceA user expresses frustration with the monthly cost of Claude Max, citing its superior output quality and deep contextual understanding of their workflow as reasons they cannot easily switch to alternatives. Despite effor…
→ View original sourceA discussion on r/LocalLLM critiques a post from OpenAI's "strategic futures" representative who likens open-weight models to communism. The author suggests this framing is a calculated attempt to generate FUD (fear, unc…
→ View original sourceThe author, u/Rich-Fruit-326, released an open‑source tool called LLM Studio, which provides an interactive 3D visualizer for transformer architectures and live inference of local LLMs. It lets users explore model struct…
→ View original sourceTL;DR — I hand-wrote a Vulkan compute engine specialized for a single model (Qwen3.6-35B-A3B) on RDNA3. It decodes at 190.7 tok/s vs llama.cpp's 132.3 on the same GGUF and the same card — 1.
→ View original sourceEULLM is an open-source, Rust-based LLM inference engine under Apache 2.0 that is Ollama/OpenAI-API compatible and ships as a single binary with no Python or Docker dependencies. It runs on Windows, Linux, and macOS with…
→ View original sourceOn an Apple M5 Max, ik_llama.cpp achieved a 20.8 % overall speedup over llama.cpp for the Qwen3.6‑35B‑A3B‑MTP model. Its gains were most pronounced in text generation, with a 31.4 % throughput increase, followed by 16.0 …
→ View original sourceA benchmark of 17 different llama.cpp commits was conducted to determine the fastest version for Qwen3.6 MTP + Vision, totaling 153 successful samples. The commit 57fe1f07c3b6 (b9620), published on June 13, 2026, emerged…
→ View original sourceThis guide demonstrates how to run the Gemma 3 12B model on a free Google Colab T4 GPU using a Cloudflare Tunnel. This method provides a solution for users with low-spec hardware, such as 8GB RAM MacBooks, by offloading …
→ View original sourceThe open_weight ecosystem is moving insanely fast this month. KwaiKAT officially confirmed they are open-sourcing the weights for KAT-Coder-Air V2.5 incredibly soon, and the model is
→ View original source