Techyon

How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus

This study investigates the "lossless" claim of Orthrus, a hybrid autoregressive-diffusion architecture designed to accelerate language model inference through parallel token generation and an intra-model consensus mecha…

→ View original source

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

Researchers introduced ModaLens, an image-swap audit framework designed to evaluate image sensitivity in report-conditioned medical vision-language models (VLMs). The study investigated whether models actually utilize vi…

→ View original source

Xi promotes open source AI zone among BRICS countries

Chinese President Xi Jinping has proposed establishing an open‑source AI zone for BRICS countries, urging deeper cooperation on artificial‑intelligence development among the bloc’s members.

→ View original source

What are the current best retail GPUs for max VRAM at a reasonable price?

The user seeks a retail GPU with high VRAM at a reasonable price to run local LLMs after their RTX 4090 failed, suspecting that recent price increases may be intended to hinder open‑source AI. They ask for recommendation…

→ View original source

I’m sending the same huge system prompt with every request

The author noticed that their LLM workflow's system prompt has expanded significantly over time, as they repeatedly added instructions and examples to address undesirable outputs and edge cases. Consequently, every reque…

→ View original source

Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'

Anthropic’s CEO warned that an AI-driven botnet “swarm” could become capable of taking over the entire internet within six to 12 months through a persistent botnet. The warning was shared in a Reddit post on r/singularit…

→ View original source

Run! GPT-2 is gonna kill us all!!

A Reddit user on r/singularity warned that GPT-2 will inevitably kill humanity. The post offers no evidence or technical analysis to support the claim.

→ View original source

Big AI sets out its terms for regulatory capture

Major AI firms have published the conditions they seek to impose on upcoming AI regulation, describing the strategy as “pace the frontier.” The move signals an attempt to steer regulatory frameworks in favor of industry …

→ View original source

K2 Horizon 36B-A4B for local AI: what the 4B-active model actually needs

The K2 Horizon 36B‑A4B language model contains 36 billion parameters in total but activates only 4 billion parameters per token during inference. This sparse activation dramatically lowers the computational and memory fo…

→ View original source

llama: add Maple 20B-A1B ternary MoE architecture (CPU) by AlexGabbia · Pull Request #27000 · ggml-org/llama.cpp

The latest pull request to ggml-org/llama.cpp introduces the Maple 20B-A1B ternary mixture-of-experts (MoE) architecture optimized for CPU inference, aiming to reduce VRAM requirements. This addition enables running the …

→ View original source

For the GPU poor. K2 Horizon 7B ranks between qwen 3.6 27B and qwen 3.6 35BA3b on the Artificial Analysis Intelligence Index.

The K2 Horizon 7B model ranks between Qwen 3.6 27B and Qwen 3.6 35B on the Artificial Analysis Intelligence Index. Early testing shows it can compile the latest llama.cpp for CUDA successfully, suggesting strong performa…

→ View original source

Qwen3.8-27B @ 100K context on an RTX 4080 16GB — ExLlamaV3 MTP results

The author ran Qwen3.8-27B with a 100K token context window on an RTX 4080 (16 GB VRAM) using ExLlamaV3 1.5.0 via TabbyAPI, measuring VRAM consumption and generation speed. Results showed the model fits within the 16 GB …

→ View original source