Techyon

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

Strata enables the 125‑billion‑parameter Qwen3.8‑Flash‑Next model to run on consumer PCs by distributing its 24,576‑expert mixture across GPU, RAM and SSD. On an NVIDIA RTX 5070 (12 GB VRAM) the Q2_0 quantization generat…

→ View original source

Homa: The end of TCP for AI clusters [video]

Homa is a novel protocol introduced in a video that seeks to replace TCP for AI cluster networking. The proposal argues that Homa can overcome TCP's limitations in high‑performance AI workloads.

→ View original source

Qwen3.5 arch implementation in FPGA fabric for 9B/27B INT4 models on relatively cheap eBay mining hardware

The author implemented the Qwen3.5 architecture in FPGA fabric to enable INT4 quantized inference of 9B‑ and 27B‑parameter language models on inexpensive eBay‑sourced mining hardware. They selected the SQRL FK33 board, w…

→ View original source

New MoE Model - Aleph-Alpha Kolibri-1 78B-A3.5B

Aleph‑Alpha has released Kolibri‑1, a 78 B‑parameter Mixture‑of‑Experts model with an active size of 3.5 B, targeting local LLM deployment. The model card shows promising benchmark results, though the author notes they h…

→ View original source

How to scale intent, quality, and artistry with AI [video]

The video explores how AI can be leveraged to scale intent, quality, and artistry in creative projects. It presents techniques for maintaining high standards while increasing output volume. The discussion focuses on prac…

→ View original source

Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability

The authors investigated whether training large language models to produce shorter chain‑of‑thought (CoT) rationales harms the transparency of those rationales. They fine‑tuned several pretrained models using three disti…

→ View original source

Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It

The study investigated source bias in LLM‑driven decision making by evaluating 12 language‑model agents across three distinct domains (product purchasing, hotel booking, and paper citation). In a controlled end‑to‑end se…

→ View original source

RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations

The authors introduce RealCompanion, a benchmark that evaluates how well an AI companion can understand a human partner by reasoning over extended, real‑world dialogues. They collected ten longitudinal relationships betw…

→ View original source

Benchmarking decision models is fun - Clef Q8 vs Jev

The user benchmarked the decision models Clef Q8 and Jev, sharing performance comparisons on the r/LocalLLaMA subreddit. The post discusses the fun of evaluating these models and invites community discussion.

→ View original source

I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens

The author trained a 3.87 B‑parameter Mixture‑of‑Experts model (1.45 B active parameters per token) from scratch on 86.5 B tokens, using a decoder‑only architecture with MoE in every layer, 32 layers, d_model = 2048, GQA…

→ View original source

I got llama.cpp inference running on the Snapdragon 8 Gen 3 Hexagon NPU from non-root Termux + Adreno OpenCL results (S24 Ultra)

The author ran llama.cpp inference on a Galaxy S24 Ultra (Snapdragon 8 Gen 3) using non‑root Termux, first getting the Adreno 750 OpenCL backend functional, though it was slower than CPU execution. They then accessed the…

→ View original source

Remove and Disable Apple Macos27 AI Models Tool

RemoveMacAI is a macOS 27 utility designed to disable Apple Intelligence features and delete associated on-device AI models. The tool uses a script (via curl) or Homebrew to execute actions: it verifies SHA-256 checksums…

→ View original source