EDN

The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking

Here is how the prefill versus generation split exposes GPU structural inefficiencies in AI processor designs.
EDN

MLPerf and the rise of latency-aware LLM benchmarking

Here is a sneak peek at the evolution of the MLPerf benchmark and how generative AI forced a radical shift in AI hardware ...
Semiconductor Engineering

Four Architectural Opportunities for LLM Inference Hardware (Google)

“Large Language Model (LLM) inference is hard. The autoregressive Decode phase of the underlying Transformer model makes LLM inference fundamentally different from training. Exacerbated by recent AI ...
Forbes

Cerebras Now The Fastest LLM Inference Processor; Its Not Even Close

The company tackled inferencing the Llama-3.1 405B foundation model and just crushed it. And for the crowds at SC24 this week in Atlanta, the company also announced it is 700 times faster than ...
Semiconductor Engineering

LLM Inference On CPUs (Intel)

“Large language models (LLMs) have demonstrated remarkable performance and tremendous potential across a wide range of tasks. However, deploying these models has been challenging due to the ...

MeMo's memory model lets teams upgrade their LLM without retraining it — and performance jumps 26%

Researchers' MeMo keeps AI memory separate from reasoning, so teams can upgrade their LLM without retraining it and see a 26% ...
Business Wire

Inception Launches Mercury 2, the Fastest Reasoning LLM — 5x Faster Than Leading Speed-Optimized LLMs, with Dramatically Lower Inference Cost

PALO ALTO, Calif.--(BUSINESS WIRE)--Inception, the company behind the first commercial diffusion large language models (dLLMs), today announced the launch of Mercury 2, the fastest reasoning LLM and ...
Crypto Briefing

MIT’s MeMo framework boosts LLM performance by 26% without retraining

MIT's MeMo framework trains a compact memory model that boosts LLM performance by up to 26.73% without retraining, with major implications for crypto AI agents.

Researchers automated LLM reasoning strategy design and cut token usage by 69.5%

Researchers from Meta and Google built AutoTTS to automatically discover optimal LLM reasoning strategies, cutting token ...
Gizmodo

ChatGPT Can ‘Infer’ Personal Details From Anonymous Text

New research shows how popular LLMs are able to accurately guess a user’s race, occupation, or location, after being fed seemingly trivial chats. Reading time 4 minutes Quiz time: If you or your ...