DeepSeek AI Releases DeepSeek-V4: Compressed Sparse Attention and Heavily Compressed Attention Enable One-Million-Token Contexts

Thinking Machines Lab has introduced a research preview of TML-Interaction-Small, a 276B parameter Mixture-of-Experts model with 12B active parameters, built around a multi-stream, time-aligned micro-turn architecture that processes 200ms chunks of audio, video, and text simultaneously — eliminating the need for external voice-activity detection harnesses. Unlike standard turn-based models that freeze perception during generation, the system runs two components in parallel: a real-time interaction model that maintains continuous full-duplex exchange with the user, and an asynchronous background model that handles sustained reasoning and tool use while sharing the full conversation context throughout. The post Mira Murati’s Thinking Machines Lab Introduces Interaction Models: A Native Multimodal Architecture for Real-Time Human-AI Collaboration appeared first on MarkTechPost.

The post DeepSeek-V4 Tackles Million-Token Context on NVIDIA HGX B200 appeared on BitcoinEthereumNews.com. Luisa Crawford May 11, 2026 18:55 DeepSeek-V4 introduces a 1M-token context window with a hybrid attention architecture, shifting the challenge to inference systems on NVIDIA hardware. DeepSeek-V4, launched by Together AI, is reshaping how AI handles ultra-long context windows by introducing a 1-million-token capacity. Rather than simply a model architecture breakthrough, V4 transforms this into a systems-level challenge, focusing on efficient inference and memory management. This innovation runs on NVIDIA HGX B200 hardware, leveraging advanced techniques like compressed Key-Value (KV) layouts, prefix caching, and hybrid attention mechanisms to address the bottlenecks of long-sequence processing. Architectural Shifts: Compressing the Token Axis At the core of DeepSeek-V4’s advancements is a hybrid attention mechanism that compresses the token axis before KV stora

DeepSeek-V4 models are open and low-cost models that also use Chinese chipmaker Huawei's AI chips for inference.

The latest set of open-source models from DeepSeek are here. While the industry anticipated the dominance of “closed” iterations like GPT-5.5, the arrival of DeepSeek-V4 has ticked the dominance in the favour of open-source AI. By combining a 1.6 trillion parameter MoE architecture with a massive 1 million token context window, DeepSeek-V4 has effectively commoditized […] The post DeepSeek-V4: The Most Powerful Open-Source Model Ever appeared first on Analytics Vidhya.