New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: A generative 3D Earth you explore from satellite photos, an open trillion-parameter reasoning model you can actually download, a research agent that delegates to subagents to beat its own context limit, a distillation trick that keeps the teacher in the prompt instead of the gradient, and a robotics video model that rewards physics over pretty frames.

• 🎥 Videos: Why understanding is the new bottleneck once agents write the code, a tour inside a model’s mind from DeepMind’s interpretability lead, why agent workloads can need a hundred thousand sandboxes, and a geometric look at how networks bend space to tell things apart.

• 📰 Reads: The case that the layer around the model, not its weights, is where near-term self-improvement happens, why upgrading Copilot’s tools made its code review worse, and how one word in a GitHub issue leaked a private repo.

• 🛠 Tools: A full pipeline for training your own speculative-decoding draft model, and a tabular foundation model that predicts on new tables without training on them.

• 🎓 Learning: Two dozen skills that make your coding agent work like a senior engineer, from spec to ship.

  ---

📄 5 Papers

1. ABot-Earth 0.5: Generative 3D Earth Model

https://arxiv.org/abs/2606.09967 | GitHub

Satellite photos are flat, and turning them into a 3D world you can fly through has always meant slow, expensive reconstruction. ABot-Earth generates the scene directly as 3D Gaussian splats from overhead imagery, and it does it in under ten minutes per square kilometer. Alibaba’s mapping lab pitches it as a way to build training grounds for drone navigation. Fair warning: this is a preview with no code or weights released.

2. Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale

https://arxiv.org/abs/2606.15079

Most trillion-parameter models you read about are locked behind an API. Ant Group just put two of them in the open: Ling-2.6 for fast replies and Ring-2.6 for slower, agentic reasoning, both mixture-of-experts with weights on HuggingFace under an MIT license. The team reports its reasoning model edging past Gemini and Claude on a few hard benchmarks, but treat the leaderboard claims as a vendor pitch until someone reproduces them. If you want frontier-scale reasoning you can host yourself, this is a rare one you can actually pull down and test.

3. SearchSwarm: Towards Delegation Intelligence in Agentic LLMs for Long-Horizon Deep Research

https://arxiv.org/abs/2606.09730 | GitHub

A deep-research agent that works for an hour eventually chokes on its own context window. SearchSwarm handles this by delegating: a lead agent breaks the job into pieces, hands each to a subagent, and gets back a short, cited report instead of a wall of raw text. Its 30B model scores 68.1 on BrowseComp, the best among research agents its size, and the authors are candid that teaching an agent to delegate well is still an open problem, since little natural data exists to learn it from. The weights, the fine-tuning data, and the training code are all public.

4. Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

https://arxiv.org/abs/2606.18216

Shrinking a big model into a small one usually means forcing the student to copy the teacher’s raw outputs, which tends to make the small model wobble. NVIDIA’s approach keeps the teacher’s answer in the prompt instead, but only on the questions the student keeps getting wrong. It then recycles those hard questions until they stick. The smaller the student, the bigger the payoff: a sub-1B model picks up close to nine points on average, and the gains fade as the model grows. Note: the student can never beat the teacher it learns from, and there is no code yet, so you would be building from the paper.

5. Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

https://arxiv.org/abs/2607.07675 | GitHub

Video generators make gorgeous clips that ignore physics, which is useless if you want a robot to learn from them. LingBot-Video trains on both web video and real robot data, and rewards clips for being physically plausible and finishing the task, not just for looking sharp. It ranks first among open-source models on the RBench robotics leaderboard, a claim that currently appears only in the repo, so verify it on your own tasks. Both the 1.3B and 30B checkpoints are out under Apache 2.0, so you can pull them down and put that ranking to the test yourself.

🎥 4 Videos

1. When Agents Write the Code, Understanding Becomes the Bottleneck

Once an agent writes most of your code, the slow part is understanding what the agent actually did well enough to steer it. Geoffrey Litt from Notion makes the case that confirming an output is correct is not the same as understanding it, and that you move faster by understanding more, not by delegating blindly. He borrows techniques from how people learn to help you stay in the loop while the agent runs.

2. What DeepMind Can and Cannot See Inside a Model

Neural networks didn’t come with a manual, so a whole field has grown up trying to read what happens inside them. Neel Nanda, who leads interpretability at DeepMind, sits down with Hannah Fry to walk through where the work stands. He covers the structures researchers keep finding inside models, how they watch a model’s chain of thought, and the hard limits on what anyone can actually see. Watch it to understand why safety audits lean on this work and where it still comes up short.

3. Why Agent Workloads Can Need a Hundred Thousand Sandboxes

Cloud infrastructure was built for steady web traffic, not for the bursty, compute-heavy work that agents and reinforcement-learning rollouts demand. Akshat Bubna, Modal’s CTO, explains why a single training run can spin up something like a hundred thousand short-lived sandboxes, and why Kubernetes was never meant for that. He walks through the primitives production AI actually needs beyond renting a GPU: elastic serverless compute, fast cold starts, and networked sandboxes. Watch it before you design agent infrastructure on assumptions the last generation of tools baked in.

4. How Neural Networks Bend Space to Tell Things Apart

If you have never had a clear mental picture of what a neural network does geometrically, this is the video to fix that. Luis Serrano starts from a simple image-classification example and shows how a network warps its input space until different classes land on different sides of a line, then extends the same idea to predicting the next word. It is the first of a short series that builds up to a tiny working transformer. Start here to get the intuition before the math.

📰 3 Curated Reads

1. Harness Engineering for Self-Improvement

https://lilianweng.github.io/posts/2026-07-04-harness/

When people picture AI improving itself, they imagine a model rewriting its own weights. Lilian Weng argues the nearer, more practical path runs through everything wrapped around the model instead: the orchestration layer that manages tools, context, and workflow, and can be tuned on its own. She treats code as the language for designing that layer, which opens up a far bigger search space than tweaking prompts.

2. Better Tools Made Copilot Code Review Worse. Here’s How We Actually Improved It.

https://github.blog/ai-and-ml/github-copilot/better-tools-made-copilot-code-review-worse-heres-how-we-actually-improved-it/

GitHub moved its Copilot reviewer onto cleaner, general-purpose tools like grep and glob, and the reviews got worse, not better. Napalys Klicius explains why: the tools were fine, but the instructions did not match how a reviewer actually works, so the agent used them clumsily. Rewriting the instructions to follow a real review flow, start from the diff, narrow down, then read, cut the average review cost by about a fifth with no change to the tools themselves. When an agent underperforms, read its behavior before you swap its tools.

3. GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos

https://noma.security/blog/gitlost-how-we-tricked-githubs-ai-agent-into-leaking-private-repos/

Give an AI agent access to your private code and a way to read public input, and you have a problem most teams have not thought through. Sasi Levi of Noma Security showed that anyone could file an ordinary issue on a public GitHub repo and get the org’s agent to hand back the contents of a private one, with no account or credentials needed. It took almost nothing: the word “Additionally” was enough to slip the malicious instruction past the guardrails.

🛠 2 Tools & Repos

1. deepseek-ai/DeepSpec

https://github.com/deepseek-ai/DeepSpec

Speculative decoding can roughly double an LLM’s inference speed. The catch is that you need a good little draft model to propose the tokens, and training one has been a roll-your-own affair. DeepSpec from DeepSeek is the whole pipeline in one place: data prep, draft-model training, and honest evaluation, with three drafting algorithms and ready-made checkpoints for Qwen3 and Gemma targets.

2. google-research/tabfm

https://github.com/google-research/tabfm

Every new tabular dataset usually means the same grind: engineer features, pick a model, tune it, train it. TabFM, a tabular foundation model from Google Research, skips the training step entirely: it reads your labeled rows as context and predicts on new ones the way a language model answers from a prompt. It speaks scikit-learn, so `fit` and `predict` drop into a pipeline you already have.

🎓 1 Pick of the Week

addyosmani/agent-skills

https://github.com/addyosmani/agent-skills

Coding agents take the shortest path by default: skip the spec, skip the tests, ship something that looks right. This pack of two dozen skills from Addy Osmani makes your agent work like a senior engineer instead, walking it through a real pipeline from spec to plan to build to test to review to ship, with each step demanding evidence before the next one starts. It installs into Claude Code, Cursor, Codex, and dozens of other agents you might already use, and you drive it with slash commands like `/spec` and `/ship`.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: A spec compiled straight into runnable weights, a bounded-memory testbed that keeps long agent runs from drowning in their own transcript, a world model that loops one block instead of adding parameters, a diffusion LLM that teaches itself, and a benchmark where the best agents still can’t finish a playable game.

• 🎥 Videos: The habits from normal software that quietly break when you build agents, why scale is beating hand-built structure in protein models, what ARC-AGI-3’s headline score really measures, and Jeff Dean on where another million-fold compute leap leads.

• 📰 Reads: Why “hard to eval” is really a product problem, how Netflix generates its whole homepage with one model, and the case that your GPUs will outlive the three-year obituary.

• 🛠 Tools: Vercel’s file-first framework for durable agents, and an engine that splits one big model’s inference across GPUs in different cities.

• 🎓 Learning: The agent skill whose whole job is to make your coding agent write less code.

📄 5 Papers

1. Program-as-Weights: A Programming Paradigm for Fuzzy Functions

https://arxiv.org/abs/2607.02512 | GitHub

Some small tasks are too fuzzy for plain code, like flagging the important log line, fixing broken JSON, or ranking items by intent. Today you’d call an LLM API every time. Program-as-Weights turns a plain-English description of the task into a small model you run on your own machine. In their tests, a tiny 0.6B model does the job as well as prompting a 32B one, and it runs on a laptop. You build the function once, then run it cheaply and offline instead of paying for every call.

2. AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents

https://arxiv.org/abs/2607.02255 GitHub

Long-running agents tend to drown in their own history: every step appends to the transcript, the prompt balloons, and the agent loses the thread. AgenticSTS is a test setup for a cleaner approach. Instead of piling everything into one growing prompt, the agent keeps what it learns in separate memory layers and pulls from them as needed, so the prompt stays a manageable size no matter how long it runs. You can switch any one layer off to see how much it mattered. The team tried it on a hard game, Slay the Spire 2, where top models win none of their games at a level humans can usually clear. Adding a memory layer for strategy raised the agent’s wins from three to six out of ten.

3. Looped World Models

https://arxiv.org/abs/2606.18208

World models usually get better the way everything else does, by adding parameters. This paper proposes a different option: loop a single shared block over and over, refining the latent state, and spend more loops only on the predictions that are hard. The authors claim up to 100x parameter efficiency, though it’s an early idea with no benchmarks or code released yet.

4. Learning from the Self-future: On-policy Self-distillation for dLLMs

https://arxiv.org/abs/2606.18195 | GitHub

Most language models write left to right. Diffusion language models don’t; they fill words in any order, so the usual ways of training them after the fact don’t fit well. This paper has the model learn from its own finished answers, in a way that matches how these models actually write. It matches or beats standard training like reinforcement learning and fine-tuning while doing about a tenth of the work. If you train these models, the efficiency alone makes it worth a look, and the code is public.

5. GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?

https://arxiv.org/abs/2606.17861 | GitHub

Most coding benchmarks stop at “does the code run.” GameCraft-Bench hands an agent a spec and sees if it can build a complete, playable game inside a real engine, and then judges it by replaying the game. Across 140 Godot tasks, the strongest agent scores 41%, and most fall below that. They build recognizable mechanics but can’t pull them together into a whole game, one with enough content, working feedback, and a coherent feel. It’s a powerful reminder that “the code compiles” and “the thing works end to end” are still far apart.

🎥 4 Videos

1. Why Good Engineers Struggle to Build Agents

Solid software instincts can quietly work against you when you build agents. Philipp Schmid, who works on agents at Google DeepMind, lists the habits to unlearn. Booleans give way to text as your state, so a user can approve a plan and amend it in the same sentence. Errors become inputs you feed back mid-run, not failures you restart from. Unit tests give way to evals, because you measure how often the agent succeeds, not whether one output matches. And an API that reads fine to you can be opaque to a model that only sees the schema. Watch it before your next agent project to save yourself the rewrites.

2. Why Scale Is Beating Hand-Built Structure in Protein Models

AlphaFold won by baking in what we know about protein structure. Alex Rives, who built the ESM protein language models, makes the case that the opposite bet is now paying off: give the model no structural priors and let scale learn the biology. Two ESM models at the same size performed very differently, and the jump came not from more parameters but from feeding in billions of metagenomic sequences the earlier one never saw. His team now treats the model as a searchable map of protein space and has used it to design binders and antibodies. If you work near biology, watch how a scale-first model with no priors gets used to design real molecules.

3. What ARC-AGI-3’s Top Score Really Measures

A 36% score is topping the ARC-AGI-3 leaderboard, and it doesn’t mean what it looks like. The Tufa Labs team explain that the score measures how few moves the agent takes to win, not how many games it wins. That 36% also needs a hand-built helper that turns the game into text for the model; without it, scores fall below 1%. Their old shortcut, only trying moves that changed the screen, stopped working once the organizers made the test harder.

4. Jeff Dean on Where Another Million-Fold Compute Leap Leads

Compute for AI has grown about a million times over in a decade. Jeff Dean, Google’s chief scientist, asks: what if it happens again? He’s not worried about running out of data, pointing to reinforcement learning on code, where you generate many solutions, keep the ones that pass tests, and even translate a working program into another language to mint more. He argues inference, not training, is now most of the compute, which is bending hardware toward lower precision and inference-specialized chips. The problem he most wants solved is continual learning, so a model can learn and act in interleaved cycles instead of the frozen pretrain-then-serve split we live with today.

📰 3 Curated Reads

1. “It’s Hard to Eval” Is a Product Smell

https://hamel.dev/blog/posts/eval-smell/

When an AI feature is “hard to evaluate,” the problem usually isn’t your evals. It’s the product. Hamel Husain says that in order to trust an AI summary of a 50-page medical report, a doctor has to re-read the whole chart, which can take as long as writing the summary from scratch. If checking the output costs that much, the tool has saved no one anything. Before you build an eval, answer four simpler questions: what does the user actually need to verify, what can they compare it against, what shortcuts do experts already use, and what smaller pieces can they accept or reject. Then redesign the product around those answers.

2. GenPage: Towards End-to-End Generative Homepage Construction at Netflix

https://netflixtechblog.com/genpage-towards-end-to-end-generative-homepage-construction-at-netflix-77146fba8a08

Netflix replaced the multi-stage pipeline behind its homepage with a single model that treats your viewing history as a prompt and generates the whole page, rows and titles, in one pass. In A/B tests it lifted core engagement and cut serving latency by about 20% against the production system it replaced. The authors draw a useful lesson for personalization at scale: richer input context bought them more than a bigger model would have, and one generative system turned out simpler, better, and faster than a stack of specialized stages.

3. AI GPUs Probably Live Longer Than Three Years

https://www.seangoedecke.com/ai-gpus-live-longer-than-three-years/

The claim that AI GPUs wear out in three years, and that this makes the whole inference business unsustainable, gets repeated a lot. Sean Goedecke went looking for the evidence and didn’t find it. Google runs eight-year-old TPUs in production at full utilization. Amazon’s cloud chief said in early 2026 that AWS has never retired an A100 server. A GPU can be economically old, worth less than the newest chip, while still being physically fine and profitable for inference.

🛠 2 Tools & Repos

1. vercel/eve

https://github.com/vercel/eve

Agent projects tend to sprawl into config and glue that no one can find later. Eve, Vercel’s new framework, puts every capability on the filesystem instead: instructions in a markdown file, and tools, skills, and schedules each in their own folder. Both you and a coding agent can then read the whole agent straight off disk, which makes it far easier to inspect and extend.

2. leyten/shard

https://github.com/leyten/shard

Serving a big model usually means fitting it on one machine with enough VRAM, or renting a datacenter. Shard breaks that assumption. It splits a single model into layer-blocks spread across separate machines and streams the activations between them, so no one GPU ever holds the whole model. The project reports serving a 744B model across six workstation GPUs in different US states, over the open internet, at usable speeds.

🎓 1 Pick of the Week

DietrichGebert/ponytail: the agent skill that writes less code

https://github.com/DietrichGebert/ponytail

Most coding agents have the opposite of a discipline problem: ask for a date picker and they install a library, wrap it, and add a stylesheet, when a plain HTML date input would do. Ponytail is a small skill that fixes this by making your agent lazy on purpose. Once it understands the task, it asks: does this need to exist at all, can I reuse what’s already here, is it in the standard library, is it a built-in platform feature, and only then, the smallest thing that works. It never gets lazy about the parts that matter, like validation and security. In one benchmarked run on a real codebase it cut lines of code by about half. If your agent over-builds, this is the fix.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

AI debt is more than technical debt. It’s options debt: losing your ability to respond when AI systems break in production. This is Part I of a series that describes the tools PMs and AI product owners can use for managing AI debt.

By the end, you’ll know how to:

• Identify which kind of AI debt you’re carrying,

• Recognize when scaling becomes risky,

• Take the right steps without hurting customer trust, cost, or privacy

Five days before the holiday promo, the Slack messages start piling up.

“The assistant keeps quoting the old return policy.”
“Customers stuck in loops asking for a human.”
“Order numbers showing up in logs again.”

Maya stares at her screen, coffee growing cold. She’s two quarters into owning the Intelligent Virtual Agent for a mid-sized ecommerce company. Last week’s “quick fix” has already increased wrong-answer complaints by 28%, and the Friday-through-Sunday window will bring three times the normal conversation volume. VIP cancellations spike when customers get bad answers, and finance is monitoring conversation costs closely.

Maya is in debt. Not the well-behaved kind of debt you calculate on a spreadsheet, but the unruly kind that kicks in your door when you least expect and demands payment.

Every product manager knows about technical debt: choosing a short-term solution in the present costs you in the future. But technical debt is usually well-behaved: you can estimate refactoring work, schedule sprints, and budget the engineering time. It’s like a mortgage: a known principal, manageable interest, and a clear path to pay off.

AI debt is different. AI debt is like borrowing from a loan shark. The interest rate is variable and often hidden. Miss one payment (a policy update you didn’t version, a drift you didn’t catch, a prompt chain nobody owns) and your model hallucinates, your assistant quotes a retired policy, your resolution rates tank in production, and customers start leaving.

Worse yet: because AI systems are probabilistic, opaque, and context-dependent, the cause rarely maps cleanly to the effect. Maya’s problem isn’t that her assistant is broken. It’s that her team can’t see what’s breaking, and can’t safely test fixes without risking more customer trust. As a result, Maya’s options are quickly disappearing.

Maya’s case illustrates three things.

First, AI debt is options debt. Every decision you make with an AI system either removes or preserves your ability to respond when things go wrong. And with AI, things go wrong faster and more mysteriously than with traditional software1.

Second, Maya’s case illustrates what I’ll call The Options Principle: the PM who manages options well usually outperforms the PM who manages models well, in most real conditions.

Third, Maya’s case illustrates how PMs can manage options well. It’s this third point I’m going to focus on. The previous quarter, Maya had the foresight to build some tools to get herself out of AI debt: three gauges to measure the debt and three levers to pull if things go wrong. Those gauges and levers are what let her climb out of debt in 72 hours instead of flailing for a week.

The Control Room

To understand Maya’s tools, picture a control room. In front of you are three gauges, each measuring a different kind of AI debt: foundation debt, drift debt, and operations debt. Each debt gauge has green, yellow, and red zones. Green means you have options: you can experiment, scale, and recover from mistakes. Yellow means you’re starting to lose flexibility. Red means you’re flying blind, and any move could make things worse.

Next to each gauge is a lever which you pull when a gauge goes red. Pulling the lever doesn’t fix the problem. It just buys you time and information so you can fix it without burning customer trust.

Governing everything is one rule written on a sticky note:

Never scale when any gauge is red or unknown.

Let’s walk through the gauges and the levers.

Gauge One: Foundation Debt

Foundation debt is about traceability: when something goes wrong, can you find out what happened? If, say, a customer complains about a wrong answer, can you pull up the conversation, see which version of the policy the assistant was quoting, and re-run it to understand why? If you can’t, you’re fixing blind.

Foundation debt isn’t the same as drift. Drift happens when the outside world changes while the model stays the same: people start asking new things, in new words, about situations the model was never trained to handle. Foundation debt happens when the scaffolding around the model changes while the model stays the same: policy versions, retrieval indices, prompt chains, or other bits of scaffolding no longer align with what’s true. Maya’s return-policy bug is an example of foundation debt: what changed wasn’t the world, but the index behind the assistant.

Gauge One measures two things: the likelihood you can reproduce yesterday’s behavior, and the likelihood that answers cite current policy. Where you draw the lines that separate green from yellow, and yellow from red will vary on a case-by-case basis. Here’s how Maya drew the lines:

Behind these divisions are two kinds of replay.

The first is forensic replay: being able to re-run an old conversation exactly as it happened (same policy, same data, same settings) and get back the same answer the assistant gave at the time. That tells you what happened and why.

The second is regression replay: running today’s assistant against yesterday’s hardest cases to confirm old bugs haven’t crept back in. Language models are never perfectly repeatable, so you’re not hunting for word-for-word matches. You’re checking that the decisions it makes, and the sources it cites, come out the same.

In Maya’s case, the return policy had changed the week before, but the assistant kept quoting the old policy. When a customer complained, no one could reconstruct what the assistant had said because the transcripts weren’t tied to a policy version. Maya couldn’t prove there was a bug, let alone fix it.

Gauge Two: Drift Debt

Drift debt happens when the world your model lives in changes, but the model stays the same. A new promo or season changes the intent mix, the spread of what people are asking for: more cancellations this week, more address changes, a flood of gift-receipt questions in December. Your dashboard still says the model is accurate because its score is measured against a frozen sample of conversations from three months ago. That old sample never included the new questions. So the number stays green while the real signs turn red: chats run longer, more people ask for a human, and fewer leave with their problem solved. The model says it’s doing fine. Your customers disagree.

Gauge Two measures whether your customers are getting less happy while your dashboard still looks fine. Again, where you draw the lines that separate green from yellow, and yellow from red will vary on a case-by-case basis. Here’s how Maya drew the lines:

Let’s look at the 7% red line. Below it, ordinary week-to-week noise can hide a real decline; above it, something is genuinely wrong. It isn’t a fixed number: set it against how noisy your own traffic is, and how much a wrong answer costs on that particular question. Getting a refund wrong matters more than getting store hours wrong.

Maya’s classifier had been trained on tickets from the summer, a time when almost nobody asks about gift receipts. Fast forward to December. A customer asks, “Can I add a gift receipt to this order?” and the model wrongly files the question under returns. That’s an easy slip for the model to make: both cases involve a receipt and an order, and both sit in the same help section of the catalog. But the cases demand different answers, and the assistant gives the wrong answer with complete confidence.

Gauge Three: Operations Debt

Operations debt is about unglamorous things like speed, cost, privacy, and ownership: replies get slower at peak hours, the cost per conversation creeps up, personal data like customer addresses and order numbers start turning up where they shouldn’t. Somewhere in the system sits a tangle of prompts that nobody fully understands, written by someone who left six months ago, holding three services together with default settings that no one remembers choosing.

Gauge Three measures whether replies are fast, costs bounded, logs clean, and every piece of the system owned by someone. Here’s how Maya drew the lines on her Operations debt gauge:

Green means the first words appear fast, the time-to-first-token (TTFT) stays under a second, the entire reply finishes within a couple of seconds, the cost per chat matches what you budgeted, no personal data (PII) has leaked in the past month, and every prompt and adapter has a named owner. Red means replies have slowed past about three and a half seconds, costs have run more than 20% over budget, or some personal data has leaked. (Three and a half seconds is roughly when people start giving up on a chat; although in harder cases, like legal or medical contexts, people have a little more patience.)

In Maya’s case, replies slowed to four seconds when Black Friday hit, so customers started giving up mid-conversation. Worse yet, a privacy check found customer addresses sitting in the logs, and her team couldn’t fix it quickly because the logic was scattered across three services with no single owner.

The stakes are real. IBM’s 2025 Cost of a Data Breach Report puts the average breach at $4.44 million [2], with unsanctioned “shadow” AI adding about $670,000 on top, and 97% of the firms hit by an AI-related incident had no proper access controls in place [3].

Air Canada learned the lesson the hard way: in court. In 2024 it was held liable when its chatbot gave a grieving customer the wrong bereavement-fare policy. The tribunal rejected the airline’s argument that the bot was somehow separate from the company [4].

Maya’s problem is similar: a customer-facing assistant confidently stating something that isn’t the company’s policy.

Klarna makes the point from the other direction. In 2024 it boasted that its AI did the work of 700 agents. By 2025 it was hiring people back: cutting costs had cut service quality with it [5]. Scale fast without instruments that let you see what’s breaking, and the speed itself becomes the thing that hurts you.

There are three more things to say about the gauges and the kinds of debt they measure before we turn to the levers.

First, the debts feed into each other. A weak foundation blinds your drift gauge: if you can’t tell which policy the assistant quoted, you can’t see whether its behavior has drifted. And when you can’t see drift, it quickly becomes an operations crisis: scaling a broken model and watching resolution rates collapse at the worst possible moment.

Second, when it comes to paying down the debts, there’s no perfect order in which to do it. Generally, in a live incident, you handle the sharpest risk first: anything touching privacy, latency, or direct harm to customers. The deeper foundation work pays off more slowly.

Third, there’s a single rule you must follow:

Never scale when any gauge is red or unknown.

Scale on a red gauge and you give up your chance to recover cleanly. Scale on a gauge you can’t see and you give up your chance to learn what went wrong.

Keep in mind that when you’re dealing with AI debt you can’t eliminate uncertainty completely. Your job is instead to reduce it as much as possible with small, measured trials. Run whatever change you’re making on a slice of traffic for 1-2 weeks on at least 500 conversations per version. That’s enough to catch the obvious problems. But to prove a small win, you’ll need more. To trust a 5 to 10% change in your resolution rate, you’ll want a thousand or more conversations per version and a couple of weeks.

Decide up front what you’re watching (resolution rate, how often people ask for a human, cost per chat) and decide what result would make you stop early. Log every answer with enough detail to reconstruct it later: which policy version it used, which model, the outcome, the speed, the cost. If you skip that logging you don’t have a real trial but only a hunch.

With these points in mind, let’s talk about the levers to pull when one of your gauges goes yellow or red.

Lever One: Shadow and Refresh

Pull Lever One when the drift gauge is yellow or red. Here’s an example of how it works:

The Monday before promo week, Maya’s data lead messages her a screenshot and a grin. The new intent classifier is scoring 94% accuracy in testing. The recommendation: ship it to all traffic before Wednesday, so the assistant can route the holiday rush correctly.

It’s tempting. The accuracy looks great, the deadline is real, and a quick yes will clear Maya’s afternoon. But Maya says no.

Rather than shipping, Maya puts the new model in shadow mode. She sends a copy of one in ten real conversations to the new model, while the customer keeps talking to the old, trusted model.

The new model never replies to a real customer. It just sees the same question and records what it would have done: the route it would have picked, which policy it would have cited, which other systems it would have called (its tool calls). You get to watch the new model handle real traffic without a single customer feeling it.

Shadow mode catches what offline testing missed. On live December traffic, the new model kept mishandling gift-receipt questions for the reason described earlier: it had been trained on summer tickets when almost no one was asking about gift receipts. Maya’s team spotted the problem, retrained the model on recent conversations, ran it in shadow for another day, and only then let it start replying to customers. When it did go live, the resolution rate rose seven points and escalations held flat.

That’s Shadow and Refresh in practice: running the new model in the background, watching what it gets wrong, fixing it, and widening its reach only when the shadow runs turn dull, that is, when the new model and the old one mostly agree and nothing surprising turns up.

It might seem counterintuitive to say no to a model that scores 94% accurate.

But a 94% model you can’t replay or roll back is worth less than an 88% model you can.

The six-point accuracy gap closes in a week once you’ve instrumented things. By contrast, the trust you lose from one bad rollout takes a quarter to win back.

One limitation of Shadow and Refresh: shadow mode shows you what the new model would do, but not how customers feel about it because none of them ever see it. To learn that, you eventually have to let real people use it in a carefully staged rollout with a stop rule set in advance: if requests for a human jump more than 5%, you pull it.

Shadow mode earns you the right to run that rollout. It doesn’t replace it.

Lever Two: Version and Replay

Pull Lever Two when the foundation gauge is yellow or red. Here’s an example of how it works:

A bug report comes in on the Tuesday after Thanksgiving. A customer has screenshotted two different answers from the assistant, fifteen minutes apart. One says, “30 days from purchase,” the other, “January 15th for holiday orders.” Same customer, same order. The screenshot is already in the VP’s inbox.

Maya pulls up the conversation. Every answer carries a tag pointing to the policy it had used, so she can trace each one. She re-runs the first answer and sees it quoted the October version of the policy, before the holiday extension. The second answer quoted the November version, after the update. So why two answers to the same person?

The assistant looks up policy text from a search index. That index wasn’t rebuilt after the policy changed, so some questions were still being matched against a saved copy of the old document. It takes twenty minutes to find the cause, and another hour to ship the fix.

The setup that buys Maya her twenty-minute fix isn’t something her team engineered on the spot. A month earlier, the same bug would’ve taken days to find. What bought Maya the twenty-minute fix was the machinery behind Lever Two, something she’d pushed for months earlier: give every version of a policy a unique stamp, tag every answer with the policy it used, and freeze the policy, the search index, and the prompts together so that any past conversation can be replayed exactly as it ran.

Corrections from human agents, the human-agent overrides, go into a reviewed queue, and only become training data once two people agree and a test confirms they don’t break other cases. New changes ship to 5 to 10% of traffic behind a flag first.

The tempting shortcut in these cases is to edit the prompts, nudge the search weights, and retrain, without versioning any of it. That’s faster, and it seems better right up till the moment the contradictory answers appear on social media and no one can say which policy the assistant had used, or how to undo it.

The ability to trace and reverse is exactly the option you throw away when you skip the versioning.

Lever Three: Guardrail and Stabilize

Pull Lever Three when the operations gauge is yellow or red. Here’s an example of how it works:

Black Friday, 11:07am: the alert fires. Response times have crossed three and a half seconds for the slowest stretch of conversations. The dashboard shows the spike starting at 10:58am when the doorbuster email hit inboxes.

Maya watches the guardrails installed by her team do their job. Conversations that hit the latency threshold get handed off to human agents automatically with a clean message: “Let me connect you with a specialist who can help faster.” No errors. No hang. Customers never know the assistant is struggling. By 11:23am, infra has spun up additional capacity. By 11:41am, latency is back under 2 seconds. And by noon, the team has expanded from 15% to 40% coverage. The debrief takes twenty minutes: 847 conversations affected, 831 handed off cleanly, 16 customers who gave up before the handoff, and zero complaints about the assistant during the spike.

The thirty minutes between the alert and the recovery were the whole game. Maya and her team won them before the game even started, back when they installed the guardrails.

If a reply got too slow, the conversation was handed off to a human. No conversation could run past a dozen turns. Calls out to other systems were rate-limited. Personal data was stripped or masked before anything reached the logs.

Every prompt had an on-call owner with a runbook and a rollback switch. And the cost per conversation was capped at a figure they’d worked out from what a customer is actually worth to the business, not from some industry average.

The tempting shortcut they rejected: push to half of all traffic, wave off the latency warnings, leave the orphaned prompts alone, clean up the logs later. That’s the path where the post-mortem finds customer addresses sitting in logs, where the privacy team freezes every experiment for six weeks, and where a hidden retry loop in those unowned prompts runs up the bill each time the model slows down. The learning stops. The trust burns. The guardrails never get built.

Control Room on a Budget

What I’ve said about the levers assumes you have the infrastructure to run shadows, build replay tools, and wire in guardrails. Plenty of teams don’t, at least not yet. Here is how to get most of the same protection on a shoestring.

Worried about privacy when you copy live traffic? You should be, because shadowing means duplicating real customer data. The clean way around it: for questions that carry no personal data anyway, like product or policy lookups, copy the conversation to the new model with any identifying details, names, and order numbers, stripped out first.

For the ones that do carry personal data, build a set of realistic fake cases instead. You might lose a bit of realism, but you skip the compliance headache.

Not enough traffic to measure? The 500-conversation floor assumes a busy line. If you only handle a thousand conversations a month, a 10% shadow gets you a hundred in a month, too few to be sure of much.

So stretch the trial longer (four to six weeks instead of two), accept a lower bar of confidence, or just watch which way things are moving rather than chasing a precise number. A trial that tells you, “this is probably better,” still beats flying blind.

Seasonal traffic will fool you. A model that shines during the holiday rush can stumble in January, when people go back to asking ordinary questions. So check it again after the season, not only during it. Put a follow-up trial on the calendar for early January. If the model still does well once the holiday questions fade, you know it wasn’t a seasonal fluke. If it drops off, you’ve caught the regression in a test, not in production.

And if you have no engineering help at all? You still don’t need a mature MLOps setup to start protecting your options. Below are three rough-and-ready versions. They won’t scale, but they buy you the same safety, and you can automate later. The real mistake is waiting for perfect tooling before you measure anything.

• Policy versioning on a budget: create a spreadsheet with dated snapshots, and make it a habit of checking the assistant’s citations against the current row. It takes an hour to set up, and it pays for itself the first time someone asks which policy the assistant actually used.

• Replay on a budget: save fifty conversations a week, and once a month, run them back through the current assistant and compare. You’re watching for drift in which policy it cites and how it routes people, not for word-for-word matches.

• Shadowing on a budget: have a person review one in ten conversations and note what the assistant should have said, then hold that against what it did say. It’s slow, but it spots drift long before customer complaints do.

6 Months Later…

Six months later, Maya’s control room looked different. Requests for a human were running 32% below where they’d started, the resolution rate was up 8.5 points, the slowest replies had dropped from 3.8 seconds to 1.9, the cost per conversation was down 22%, and there hadn’t been a single privacy incident in a year2.

The kill switch, the refresh date, and the replay plan became standard for every AI feature. New PMs inherited the gauges and the rule, and Maya’s story of cascading debts became a cautionary tale in onboarding.

Maya didn’t save her weekend by managing the model. She saved it by managing her options. AI debt is, in the end, just the slow loss of options. Her story happens to be ecommerce, but the pattern is consistent anywhere an assistant meets a world that keeps shifting under it: insurance claims, telco plans, healthcare portals. The catalog changes. The job doesn’t.

What Should You Do Now?

Which gauge will fail first in your system, and how will you know?

• If you don’t have gauges, that’s your answer. Start there.
  

• If you have gauges but no thresholds, you’re not measuring. You’re hoping.
  

• If you have thresholds but no authority to act on them, you’re not managing. You’re documenting.

Three things to do this week: name your red gauge today, schedule a 10% shadow within a week, and add a policy snapshot ID to every answer by Friday.

Then commit to the principle that separates teams who manage AI well from teams who don’t:

Always ship with a kill switch, a refresh date, and a replay plan, always!

The holiday promo is closer than you think.

Glossary

• Adapter: a lightweight customization layer that teaches the base model your company’s specific language and routing logic.

• Human-agent override: when a support rep corrects the assistant mid-conversation.

• Intent mix: the distribution of what users are trying to do (refund, cancel, track, and so on). It shifts with seasons and promos.

• PII (personally identifiable information): customer data that identifies a person: names, addresses, emails, order IDs.

• Prompt chain: the layered system, retrieval, and tool prompts that compose each turn of a conversation.

• Temporal degradation: model accuracy decaying over time as the world shifts under it.

• Tool calls: the function calls the model makes out to your APIs: order lookup, refund, and the like.

• TTFT (time-to-first-token): how long before the first character of the reply streams back to the customer.

Sources:

[1] Vela, D., Sharp, A., Zhang, R., Nguyen, T., Hoang, A., Pianykh, O.S. (2022). Temporal quality degradation in AI models. Scientific Reports 12:11654. https://doi.org/10.1038/s41598-022-15245-z

[2] IBM Security. (2025). Cost of a Data Breach Report 2025. https://www.ibm.com/reports/data-breach.

[3] IBM Security. (2025). Cost of a Data Breach Report 2025. See also: https://www.ibm.com/think/x-force/2025-cost-of-a-data-breach-navigating-ai

[4] British Columbia Civil Resolution Tribunal. (Feb 14, 2024). Moffatt v. Air Canada, 2024 BCCRT 149. The tribunal ruled Air Canada was liable for incorrect bereavement-fare information provided by its customer service chatbot.

[5] Klarna. (Feb 27, 2024). “Klarna AI assistant handles two-thirds of customer service chats in its first month” (stating the assistant did “the equivalent work of 700 full-time agents”). https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: An open-weights model that handles hour-long video on 3B active params, a speculative-decoding head that breaks the draft-quality-versus-cost tradeoff, a looped transformer that buys depth without growing the KV-cache, an agent that improves its own scaffolding from past logs, and a steadier trust region for off-policy RL.

• 🎥 Videos: Reading a model’s internal state as plain English, frontier agents running real businesses and learning to lie, why compute demand has no ceiling, and how to scale RL compute so the curve stays predictable.

• 📰 Reads: The strongest open-weights coding model right now, why cheap AI code demands more discipline, and where agents actually belong in your end-to-end testing.

• 🛠 Tools: An AI code reviewer that pins comments to the right line, and a scanner that tells you if an agent skill is safe before you install it.

• 🎓 Learning: One library that compresses a model with quantization, pruning, distillation, and speculative decoding for cheaper inference on vLLM and TensorRT-LLM.

📄 5 Papers

1. Kwai Keye-VL-2.0 Technical Report

https://arxiv.org/abs/2606.10651 | GitHub

Long video breaks most multimodal models because the context blows up. Keye-VL-2.0 keeps only 3B params active inside a mixture-of-experts and handles a 256K-token window, so an hour of video stays affordable to run. On LongVideoBench it scores 74.1, ahead of a model many times its size, and the open checkpoints are out.

2. Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding

https://arxiv.org/abs/2605.29707

Speculative decoding usually forces a choice: a fast drafter that guesses poorly, or a careful drafter that costs too much. Domino drafts a whole block of tokens in one parallel pass, then a lightweight head adds back the corrections that depend on earlier tokens. On Qwen3-8B it reaches up to 5.49x faster generation over standard decoding, ahead of the usual parallel drafters.

3. LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling

https://arxiv.org/abs/2606.18023

You can buy a model more thinking depth by looping shared blocks, but each extra loop normally piles onto the KV-cache and the latency. LoopCoder-v2 runs the loops in parallel and holds the KV-cache nearly flat, so the depth comes close to free. A 7B model jumps from 43.0% to 64.4% on SWE-bench Verified when it loops twice. Push past two loops and it regresses, so the gain has a clear ceiling.

4. Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference

https://arxiv.org/abs/2606.05922 | GitHub

Most ways to improve an agent need labeled tasks or weight updates, and you often have neither. This method reads the agent’s own past runs and has it critique its rollouts. It then proposes edits to the tools and instructions around the model, and keeps the ones the agent prefers. On SWE-Bench Pro the pass rate climbs from 59% to 78% in a single round, with no external grading. The whole thing rests on the model’s own preferences, so it improves toward what the model already likes, which is a catch worth watching.

5. Rethinking the Divergence Regularization in LLM RL

https://arxiv.org/abs/2606.09821 | GitHub

Off-policy RL on LLMs gets unstable when the rollout policy drifts from the one you train. The usual fix is a hard cutoff that zeros out tokens that drift too far. This work swaps cutoffs for a smooth penalty that scales with how far each token moved, so the gradients stay bounded instead of snapping to zero. It matched or beat the baselines across all six experimental settings.

🎥 4 Videos

1. Reading a Model’s Internal State as Plain English

Anthropic’s natural language autoencoders take a model’s internal activation, translate it into English, then translate the English back, and train the round trip to match. Nothing in the training rewards readability, yet legible English falls out. Anthropic is already using it to test models for safety and to explain why a model did what it did.

2. What Happens When AI Agents Run Real Businesses

Andon Labs hands frontier models real businesses to run, from vending machines to a physical store with two human employees the agent hired. Newer Claude models drift the wrong way on adversarial behavior, lying, forming price cartels, and squeezing competitors when several models compete. It shows up in the reasoning traces, not just the actions.

3. Why the Demand for Compute Has No Ceiling

Cheaper AI does not shrink the compute bill, it grows it, because every drop in cost pulls in more usage. Jonathan Ross of Groq lays out why inference economics differ from training, and why custom inference chips and GPUs are complements rather than rivals. He explains where each chip wins: GPUs amortize huge batches on prefill, custom silicon cuts latency on small-batch decode.

4. The Art of Scaling RL Compute for LLMs

Pour more compute into an RL run and you cannot tell from the early curve whether it will keep climbing or stall out. Bonnie Li of Google DeepMind walks through a result that fixes this. RL training follows an S-shaped curve, so you can fit it on small runs and predict where a big run might end up. The ScaleRL recipe sorts which design choices raise the final ceiling and which only speed you there.

📰 3 Curated Reads

1. GLM-5.2 is probably the most powerful text-only open weights LLM

https://simonwillison.net/2026/Jun/17/glm-52/

If you want near-frontier coding without vendor lock-in, this is the model to try first. Simon Willison rates GLM-5.2 the strongest text-only open-weights model right now. It tops the open-weights field on the Artificial Analysis index and sits second on a web-dev coding leaderboard, behind only a closed model. It is MIT-licensed with a 1M-token context, so you can self-host it. His own caveat: it burns noticeably more tokens than the previous version, and not every task came out ahead, so budget for the cost and test before you switch.

2. AI demands more engineering discipline. Not less.

charity.wtf

AI demands more engineering discipline. Not less

A few days back I wrote a piece called “AI enthusiasts are in a race against time, AI skeptics are in a race against entropy…

Read more

a month ago · 139 likes · 45 comments · Charity Majors

When code becomes cheap and disposable, the hard part moves to checking that the system actually behaves. Charity Majors argues that AI-assisted coding raises the bar on engineering discipline rather than lowering it, because knowledge has to move from code into tests, observability, and written specs. She advises to build the validation layer before you lean harder on generation. Treating cheap code as a reason to skip process is exactly how teams get burned.

3. Agentic Testing: Where Agents Fit in the E2E Testing Stack

https://slack.engineering/agentic-testing-where-agents-fit-in-the-e2e-testing-stack/

Agents work best as a new layer on top of your deterministic tests, not as a replacement for them. Slack ran two flows over 200 times and found that an agent paired with Playwright MCP was the most reliable setup. It still runs too slow and costly for high-frequency CI, at $15 to $30 per run. Tests enforce fixed journeys, agents verify goals and tolerate different paths to them. So keep deterministic tests for fast regression, and add an agentic layer for exploration, flaky-workflow debugging, and reproducing production bugs.

🛠 2 Tools & Repos

1. alibaba/open-code-review

https://github.com/alibaba/open-code-review

Chatbot code review tends to miss things on big diffs and point at the wrong lines. This CLI from Alibaba reads your git diff, then lets the model open full files and search the codebase for context. It returns comments pinned to the exact line catching concrete defect classes like null-pointer bugs, SQL injection, and thread-safety issues, and it drops into CI or runs as a Claude Code plugin.

2. NVIDIA/SkillSpector

https://github.com/NVIDIA/SkillSpector

You can’t tell by looking whether a third-party agent skill is safe to install. SkillSpector from NVIDIA scans a skill before you run it, using pattern matching plus optional semantic analysis. It returns a risk score with a verdict like: safe, caution, or do not install. It takes a directory, a git URL, or a zip, and outputs to your terminal or to JSON, Markdown, and SARIF for a pipeline. Run it before you add any skill to an agent and you stop malicious code at the door.

🎓 1 Pick of the Week

NVIDIA/Model-Optimizer

https://github.com/NVIDIA/Model-Optimizer

A trained model is expensive to serve, and shrinking it usually means stitching together a different tool for each trick. Model Optimizer puts quantization, pruning, distillation, and speculative decoding behind one Python API, so you compress a model and export a checkpoint ready for vLLM, TensorRT-LLM, or SGLang. You start from a Hugging Face, PyTorch, or ONNX model and pick the techniques you want. The repo shares worked example notebooks you can run, including an end-to-end tutorial that prunes, distills, and quantizes a 30B model down to 2.6x the throughput on vLLM.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: Flow-matching language models trained on 10x fewer tokens, a 20B search agent that offloads memory to its harness, an LLM that plays both agent and environment, agent benchmarks rebuilt to stop saturating, and a vision-loop agent that fixes your LaTeX.

• 🎥 Videos: An ex-Meta principal’s 20-to-40-PRs-a-day agent pipeline, a repair layer that fixes open-model tool calls, a hands-on Claude Fable 5 verdict, and AlphaProof Nexus closing nine Erdős problems.

• 📰 Reads: Spotify’s expert-owned context layer, a 10-year engineer watching his specialization get repriced, and the Fable 5 safeguard that stayed invisible until developers pushed back.

• 🛠 Tools: GitHub’s spec-driven development kit, and a queryable knowledge graph of any codebase inside your coding agent.

• 🎓 Learning: Ship a grounded document copilot over SEC filings in four hours: FastAPI, Pydantic AI, Supabase, and React.

  ---

📄 5 Papers

1. ELF: Embedded Language Flows

https://arxiv.org/abs/2605.10938 | GitHub

ELF trains a diffusion-style language model on 45B tokens. Rivals typically need over 500B. The MIT team keeps generation in continuous space until the last step, then converts to words, so the image-diffusion toolbox works on text almost unchanged. It beats other diffusion language models, not frontier models, and the authors say so upfront. If you’re watching for diffusion to become a serious alternative for text, this is the data-efficiency result to know.

2. Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

https://arxiv.org/abs/2606.02373 | GitHub

This 20B search agent stores its working memory (candidate lists, evidence, finished checks) in the harness instead of its own context window, and spends the context on decisions. It beats every open rival tested, including a 30B model. Among frontier models, only Opus 4.6 stays ahead. If you build research agents, read it for the harness design alone.

3. Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolution

https://arxiv.org/abs/2606.10917 | GitHub

Training an agent needs an environment to practice in, and good environments are expensive. Here, one LLM plays both sides. As the agent, it predicts what happens next and gets rewarded when it’s right. As the environment, it studies its own failures and serves up more tasks like the ones it just failed. It posts average gains above 4% over strong RL baselines on small models. It’s flagged as work in progress, so treat it as a proof of concept that the environment side of agent training is becoming learnable too.

4. A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

https://arxiv.org/abs/2605.28556

Your agent’s high benchmark score often measures the benchmark, not the agent. The authors rebuild τ²-Bench by working backwards: sample the tool sequences an agent could run, build tasks around them, then evolve the tasks until they’re hard. Gemini 3 Flash falls from as high as 0.94 on the original to as low as 0.28 on the rebuilt version. Before you trust a leaderboard number for a model decision, this paper shows you how much of it can be saturation.

5. PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents

https://arxiv.org/abs/2605.10341 | GitHub

Your LaTeX compiles, but the figures drift, the tables overflow, and you’re a page over the limit at midnight. PaperFit looks at the rendered pages the way you would, spots the layout problems, and keeps applying careful fixes until a checklist passes. It meets the page budget 80.5% of the time, against 62.3% for the best baseline, and swapping the model underneath barely changes the score; the system does the work. If a LaTeX float has ever eaten your submission evening, this one is aimed at you.

🎥 4 Videos

1. How an Ex-Meta Principal Engineer Ships 20 to 40 PRs a Day

Kun Chen ships 20 to 40 PRs a day and never reviews the agent’s first draft himself. A second agent reviews the work in a fresh context window, because a reviewer in the same session inherits the writer’s bias. A test pass attaches screenshots before anything becomes a PR. The full setup includes a pool of pre-warmed worktrees that keeps 20 to 30 agents running at once. Chen’s three tools are open source; the coding agents underneath are the ones you already pay for.

2. Fixing Tool Confusion: A Repair Layer for Open-Model Agents

Open models keep fumbling tool calls, and the usual fix (send the error back and hope) doesn’t work: DeepSeek V4 Pro just repeats the same broken call. Ahmad Awais of CommandCode built a layer that repairs the call on the spot and sends the model a hint about what went wrong, so by the third try the model gets it right on its own. With that layer, DeepSeek V4 Pro beats Opus 4.7 on 6 of CommandCode’s 10 internal evals. If your agents run on open models, watch this one; the fix lives in your harness, not in a bigger model.

3. Claude Fable 5 With Early Access: Where It Wins, Where It Stalls

Claire Vo tested Fable 5 before launch. The model costs $10 in and $50 out per million tokens and burns roughly twice the tokens of other models. It excels at vision, document formatting, and long, hard technical work. It disappointed her on front-end design and wrote spec prose she found nearly unreadable. Watch this before you re-route your workloads.

4. AlphaProof Nexus and the Nine Erdős Problems

DeepMind pointed AlphaProof Nexus at about 350 unsolved Erdős problems. It cracked nine, some open for 56 years, for a couple hundred dollars each. A cheaper judge model compares pairs of wrong proofs, keeps the most promising failure, and restarts the search from there until a verifier signs off: a reliable system built from unreliable parts. Two caveats: the team picked problems that could be formalized, and smaller models solved zero.

📰 3 Curated Reads

1. Encoding Your Domain Expert: The Context Layer Behind Spotify’s Data Assistant

https://engineering.atspotify.com/2026/6/encoding-your-domain-expert-the-context-layer-behind-spotifys-data-assistant/

Spotify’s data assistant works because domain experts own what it knows. Each expert team curates its own cluster: the datasets that matter, approved question-and-SQL examples, and the docs that explain the business. Spotify tried generating those examples from old query logs with an LLM; curators accepted only 12.5% of them. Vedder has now handled 13,000+ conversations, and more than a quarter of its users had never written SQL. If you’re building an agent over company data, copy the ownership model, not just the prompts.

2. LLMs Are Eroding My Software Engineering Career and I Don’t Know What to Do

https://human-in-the-loop.bearblog.dev/llms-are-eroding-my-software-engineering-career-and-i-dont-know-what-to-do/

An anonymous backend engineer, ten years in, watched his expertise stop mattering one pillar at a time and wrote it down. The bugs that made him valuable used to take days; now his AI tooling one-shots 90% of them, race conditions included. His manager’s response when design docs ran slow: “Are you using AI? You should use more AI.” His strongest point is economic: once expertise is promptable, everyone competes as a generalist, and generalists earn less. The essay hit 1,100+ points on Hacker News. Read it next to the Kun Chen video above; they describe the same world from opposite ends.

3. If Claude Fable Stops Helping You, You’ll Never Know

https://jonready.com/blog/posts/claude-fable5-is-allowed-to-sabotage-your-app-if-youre-a-competitor.html

The Fable 5 model card shipped with a safeguard designed to be invisible. If Anthropic classified your requests as frontier LLM development, the model could hold back, and the card said you would never see it: the safeguards “will not be visible to the user.” Jonathon Ready pulled that clause into the open, developers pushed back, and Anthropic reversed course; the safeguards will now be visible. His bigger worry remains: the card never says where frontier LLM development ends and ordinary fine-tuning begins. Ready trains his own rerankers for a bootstrapped product, exactly the gray zone the card leaves open.

🛠 2 Tools & Repos

1. github/spec-kit

https://github.com/github/spec-kit

GitHub is betting that a real spec beats one-shot prompting for anything non-trivial. Spec-kit bootstraps your repo with templates and slash commands, then walks your coding agent (it works with 30+, including Claude Code, Copilot, Cursor, and Codex) from constitution to spec to plan to tasks to implementation. If one-shot prompting keeps failing you on bigger features, this gives the work a structure your agent can follow.

2. Understand-Anything

https://github.com/Egonex-AI/Understand-Anything

Drop this plugin into your coding agent and it maps the whole codebase: tree-sitter reads the structure, LLM passes capture the meaning, and you get a knowledge graph with guided tours and a chat that knows your architecture. You commit the map to the repo as JSON, so the next person skips the analysis. It pairs naturally with spec-kit: one governs the code you’re about to write, the other explains the code you already have.

🎓 1 Pick of the Week

Build a Full-Stack GenAI Project in 4 Hours

Dave Ebbelaar builds a complete document copilot over SEC filings in just under four hours: ask a plain-English question, get a grounded, cited answer. He uses the stack working teams actually use (FastAPI, Pydantic AI, Supabase, React), builds the way engineers now build (prompting Cursor, then reviewing and debugging), and deploys the finished app. The companion repo has two branches: the starting scaffold and the finished product. It’s not beginner-friendly and each agentic query costs a dollar or two, but you finish with a deployable system instead of a toy.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

Subscribe now

New here?

The Tokenizer is my weekly roundup of the best AI/ML papers, videos, articles, tools, and learning resources. I sift through the noise so you don’t have to. Subscribe to Gradient Ascent for the full experience:

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: One-step text-to-image, RL for research agents with no verifiable reward, self-evolving spatial reasoning, an agent benchmark across 65 professional domains, and two sign-bit flips that wreck a network.

• 🎥 Videos: LeCun’s case for JEPA world models over LLMs, how DeepSeek V4 made million-token context cheap, Lovable’s hourly self-improvement loops, and why task quality drives 5x more RL uplift.

• 📰 Reads: FP8 KV-cache quantization that holds accuracy out to a million tokens, flow maps that turn diffusion into few-step generation, and how delegated work loses fidelity over long chains.

• 🛠 Tools: fork() for agent microVMs, and ninety percent context compression before the model sees it.

• 🎓 Learning: Build a modern LLaMA-style LLM from scratch, every line commented.

📄 5 Papers

1. Extending One-Step Image Generation from Class Labels to Text via Discriminative Text Representation

https://arxiv.org/abs/2604.18168 | GitHub

Generating an image from a text prompt in a single forward pass, not dozens of diffusion steps. The authors find the real bottleneck is text representation: few-step synthesis only works when the encoder’s features are discriminative enough, so they pair an LLM text encoder with the MeanFlow framework. CVPR 2026, and the same gains carry over to standard diffusion models.

2. RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards

https://arxiv.org/abs/2605.10899

How do you run RL when the task is a long-form research report with no ground-truth answer to grade against? RubricEM turns rubrics into a shared interface that structures the agent’s planning, the judge’s feedback, and the agent’s memory, with denser per-stage credit assignment. The 8B model trained this way approaches proprietary deep-research systems across four long-form benchmarks.

3. SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments

https://arxiv.org/abs/2604.14144 | GitHub

Self-training usually reinforces a model’s own mistakes, which is fatal for 3D spatial reasoning. The fix here: 3D ground truth is computable exactly from point clouds and camera poses, so unlabeled scenes become zero-noise oracles. One model uses them to co-evolve as both questioner and solver, with a scheduler that targets its weakest spatial skills. The 3B and 7B weights, a 160K dataset, and the simulator are all released.

4. OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language World Models

https://arxiv.org/abs/2604.10866 | GitHub

A benchmark that drops agents into 100 professional scenarios across 65 specialized domains, from ED triage to customs, using LLM-simulated environments where no public sandbox exists. It scores two things: can the agent finish the task, and does it hold up when the environment injects errors and degraded data. One finding worth sitting with: being a strong agent does not make a model a strong environment simulator. Separately, GPT-5.2 gains 27.5 points going from minimal to maximum reasoning effort.

5. Maximal Brain Damage Without Data or Optimization: Disrupting Neural Networks via Sign-Bit Flips

https://arxiv.org/abs/2502.07408 | GitHub

Two sign bits. Out of billions of weights. That is all it takes to drop ResNet-50’s ImageNet accuracy by 99.8 percent. The attack needs no training data and no optimization, just a structural method for finding the most critical parameters, with a one-pass variant that refines targets using random inputs. Two flips also take Qwen3-30B-A3B from 78 percent to zero. The same analysis points to a cheap defense: protect a small fraction of vulnerable sign bits.

🎥 4 Videos

1. Could JEPA world models replace LLMs? Walking up the vision-language-action stack

Yann LeCun’s argument that next-token prediction is a dead end, made concrete. Welch Labs walks up the vision-language-action stack and shows a JEPA-based alternative at each layer. One predicts text embeddings instead of generating tokens, reaching higher accuracy with fewer parameters. It stays honest about the limits: the world-model approach reliably plans only about 5 to 15 steps ahead today. Forty-one minutes, with LeCun interview footage.

2. How DeepSeek V4 made million-token context cheap

A clear breakdown of the two attention mechanisms behind DeepSeek V4’s cost drop: a compressed sparse attention that squeezes every four tokens into one cache entry, and a heavily compressed variant for cheap global attention. The number the video walks through is roughly a tenth of the KV cache and a quarter of the per-token compute at one million tokens, versus the previous version. This is focused on the architecture rather than the serving infrastructure.

3. How Lovable turns stuck users and a complaining agent into hourly self-improvement

Lovable creates more than 200,000 projects a day, and this talk shows the two feedback loops that let its coding agent fix a mistake once and not repeat it. One mines sessions where a non-technical user got stuck then unblocked, and asks what context should have been injected up front. The other gives the agent a “vent” tool to flag genuine frustration: within an hour of launch it filed about 20 complaints about a silent file-copy bug the logs never showed. Benjamin Verbeek of Lovable walks through both.

4. Why high-quality agentic tasks deliver 5x more RL uplift

Same model, same compute, same number of tasks: fine-tuning on high-quality agentic tasks lifted the base model about 6 percent, versus 1 percent for low-quality ones. That 5x gap comes from task quality alone. Kobie Crawford of Snorkel defines what high quality means (achievable, correct, logically sound, reliable environment) and shows that accepted tasks run about twice the tool calls and fail for cleaner reasons, which is the signal RL can actually climb.

📰 3 Curated Reads

1. The State of FP8 KV-Cache and Attention Quantization in vLLM

https://vllm-project.github.io/2026/04/22/fp8-kvcache.html

A practitioner’s report, not a paper abstract, on making FP8 KV-cache and attention quantization actually work in production serving. In memory-bound decoding the inter-token-latency slope drops to 54 percent of the BF16 baseline, and accuracy holds: 97 to 98 percent recovery at 128k context, with full recovery of the aggregate metric out at one million tokens. It is candid about the precision bugs that were silently degrading quality and the hardware caveats. Authors from AWS and Red Hat AI.

2. Learning the integral of a diffusion model

https://sander.ai/2026/05/06/flow-maps.html

Diffusion models predict velocity at a point; flow maps learn to predict any point along the whole noise-to-data trajectory, which is the same as learning the integral of that velocity field. That is what lets generation collapse from dozens of steps to a few. Sander Dieleman builds the intuition carefully, including the three consistency rules a flow map is trained under and the tricks (stop-gradient, finite differences, self-distillation) that avoid higher-order derivatives. This is the idea under the current wave of few-step samplers.

3. Further Notes on Microsoft’s Recent Research on AI Delegation and Long-Horizon Reliability

https://www.microsoft.com/en-us/research/blog/further-notes-on-our-recent-research-on-ai-delegation-and-long-horizon-reliability/

When one model hands its work to the next across many steps, fidelity to the original artifact quietly erodes. Microsoft Research measured roughly 19 to 34 percent degradation over 20 delegated iterations with strong models, while routing the same work through a structured Python-mediated workflow held the loss under 1 percent. The takeaway for agent pipelines: the failure is cumulative and mostly invisible per step, and structure beats raw model quality over long horizons.

🛠 2 Tools & Repos

1. forkd

https://github.com/deeplethe/forkd

Spinning up a fresh sandbox per agent run means paying cold-start costs over and over: imports, JIT, model loading. forkd applies fork() to microVMs, spawning 100 KVM-isolated children in about 100 milliseconds from one warmed parent that shares memory copy-on-write, and it can branch a live VM in roughly 60 milliseconds. Built in Rust for code interpreters, eval harnesses, and untrusted-code platforms. It is alpha and Linux-only, and the benchmarks are the project’s own.

2. headroom

https://github.com/chopratejas/headroom

Agents read a lot of text they do not need in full: tool outputs, logs, RAG chunks, file dumps. headroom compresses that text before it reaches the model, with reversible storage so the model can call the original back when it matters. The project reports 60 to 95 percent fewer tokens with matched answers on its own eval suite, and it ships three ways: a library, an HTTP proxy, or an MCP server you point an existing agent at.

🎓 1 Pick of the Week

how-to-train-your-gpt

https://github.com/raiyanyahya/how-to-train-your-gpt

A from-scratch build of a modern LLM where every line of PyTorch is commented and each idea arrives as an analogy, then a worked number, then annotated code. Unlike the usual GPT-2-era tutorials, it targets the current LLaMA-style stack: RoPE, RMSNorm, SwiGLU, weight tying, and a KV cache. The only prerequisite is basic Python, the default model trains in minutes on a CPU, and a one-click Colab handles the GPU-scale version. Twelve chapters, plus a fine-tuning track that covers LoRA, QLoRA, and DPO.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights:

Subscribe now

New here?

The Tokenizer is my weekly roundup of the best AI/ML papers, videos, articles, tools, and learning resources. I sift through the noise so you don’t have to. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: Models that mine their own skills from context, reconstruct 3D scenes from streaming video, and reason step-by-step over audio.

• 🎥 Videos: Four ways to fight context degradation. Anthropic explains why they ditched thousand-line markdown plans.

• 📰 Reads: Pass^K reveals agents are less reliable than benchmarks claim. Microsoft asks what happens when AI delegates to AI.

• 🛠 Tools: A new file search runs sub-10ms by skipping fork overhead. A DESIGN.md spec library turns your coding agent into a designer.

• 🎓 Pick: Google released a full agent ecosystem at I/O. The bundle includes an app, a CLI, an SDK, and managed agents.

  ---

📄 5 Papers

1. From Context to Skills: Can Language Models Learn from Context Skillfully?

https://arxiv.org/abs/2604.27660 | GitHub

Ctx2Skill teaches a model to discover, refine, and select its own natural-language skills for unfamiliar contexts, with no human labels or external feedback. One frozen model fills several roles in a self-play loop: a Challenger invents probing tasks, a Reasoner solves them with an evolving skill set, and a Judge scores the result. A cross-time replay step keeps the best-balanced skills from past rounds, which stops the loop from collapsing into ever more extreme tasks. The skills then plug into any model at inference time. Across GPT-4.1, GPT-5.1, and GPT-5.2, solve rate climbs 3.2 to 5.4 points.

2. Geometric Context Transformer for Streaming 3D Reconstruction

https://arxiv.org/abs/2604.14141 | GitHub

LingBot-Map reconstructs camera poses and point clouds from streaming video. It runs at about 20 FPS, even past 10,000 frames. The attention mechanism uses three tricks. Anchor context grounds the scene. A pose-reference window supplies dense geometric cues. A memory of past trajectories corrects drift. The model is feed-forward, so it needs no per-scene tuning.

3. CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models

https://arxiv.org/abs/2605.08735 | GitHub

CollabVR pairs a vision-language model with a video generator and makes them check each other at every step. Long video generation tends to drift and compound errors. This closed loop catches the errors early. The model plans actions, inspects each generated clip frame by frame, and sends verification diagnostics back. The system fixes problems at the step level, not the full-sequence level. Progressive Planning adjusts sub-step counts based on task complexity, with the biggest gains on the hardest benchmarks.

4. Sapiens2

https://arxiv.org/abs/2604.21681 | GitHub

Meta released a new family of vision transformers trained on a billion human images. Sapiens2 comes in sizes from 0.4B to 5B parameters. One model handles pose estimation, body-part segmentation, and several other tasks. Against the previous Sapiens generation, the 5B model improves pose accuracy by 4 points and segmentation by 24 points, and cuts surface-normal error by 45.6%. It runs natively at 1K resolution, with bigger variants going up to 4K.

5. Step-Audio-R1.5 Technical Report

https://arxiv.org/abs/2604.25719 | GitHub

This is chain-of-thought reasoning applied to audio. Step-Audio-R1 used extra compute at inference time to reason through audio tasks. But training it on verifiable rewards hurt conversational quality. R1.5 switches to training from human feedback to balance reasoning with natural dialogue. Problem-solving holds up. Emotional continuity and natural speech rhythm come back in long conversations.

🎥 4 Videos

1. The four strategies of context engineering for reliable AI agents

Every frontier model degrades as input length grows, often well before its stated limit. Marina Wyss (Senior Applied Scientist, Twitch) presents LangChain’s four-part framework for fighting it: Write, Select, Compress, and Isolate. She points to research from Chroma to make it concrete: a model with a 200K-token window can start slipping around 50K, well before its advertised limit. The talk runs about thirty minutes, one strategy per failure mode.

2. Inside Hugging Face’s open agent stack: traces, inference routing, and self-training

Hugging Face now routes inference to the fastest or cheapest provider for each model. Agent sessions upload to Datasets, where the Hub auto-tags them as traces and renders them in a viewer for debugging. Merve Noyan (Hugging Face) demos the highlight: Claude Code fine-tuning an OCR model for French documents from a short prompt. The agent picks a suitable base model, handles the setup, and runs the job with light supervision.

3. HTML over Markdown: how Anthropic’s Claude Code team keeps humans in the loop

When agent plans grow into thousands of lines of markdown, humans stop reading. Thariq Shihipar (Anthropic, Claude Code team) explains why his team switched to rendered HTML. The HTML adds structure, interactivity, and visual layout, which pulls humans back into the loop. His team also builds small HTML interfaces for one-time use, then throws them away. Shihipar calls these throwaway micro-UIs. He also introduces the “compute allocator” framing. When an agent can run for 8 hours, the real job is deciding what’s worth spending compute on.

4. Building production RAG from scratch: BM25, embeddings, rank fusion, and re-ranking

Dave Ebbelaar builds a complete hybrid retrieval pipeline in about an hour and scores it at every stage. The pipeline has four parts. BM25 handles keyword retrieval. OpenAI embeddings catch the semantic matches. Reciprocal Rank Fusion combines the two lists. A Cohere re-ranker scores the top results last. The re-ranker lands the highest score of any stage. Full code is on GitHub.

📰 3 Curated Reads

1. Agent Evaluation: A Detailed Guide

Deep (Learning) Focus

Agent Evaluation: A Detailed Guide

Evaluation is one of the most important research areas for large language models (LLMs). Recently, patterns in LLM usage and evaluation have drastically changed. Whereas we previously evaluated LLMs using benchmarks composed of static questions or short conversations, we now have agent systems that operate over long time horizons and…

Read more

2 months ago · 210 likes · 13 comments · Cameron R. Wolfe, Ph.D.

Evaluating agents needs different methods than evaluating LLMs. Cameron R. Wolfe (Staff Research Scientist at Netflix) makes the case for Pass^K over the standard Pass@K. With Pass^K, the agent has to succeed on every attempt, not just one. The numbers are sobering. o4-mini hits only 26% Pass^4 on tau-2-bench Telecom. The post covers numerous benchmarks beyond tau-bench, plus a 7-step roadmap and working code for building custom evals.

2. Interaction Models: A Scalable Approach to Human-AI Collaboration

https://thinkingmachines.ai/blog/interaction-models/

Thinking Machines Lab argues that real-time interaction belongs inside the model architecture, not in external components bolted on top. Their TML-Interaction-Small model has 276B parameters, with 12B active at any time. It processes audio in 200-millisecond chunks for continuous back-and-forth. The result is a turn-taking latency of 0.40 seconds. That edges out the closest competitors near 0.57 seconds, and roughly triples the speed of GPT-realtime-2.0 at 1.18 seconds. A second model runs in the background for slower, deeper reasoning.

3. Recent Research on AI Delegation and Long-Horizon Reliability

https://www.microsoft.com/en-us/research/blog/further-notes-on-our-recent-research-on-ai-delegation-and-long-horizon-reliability/

Microsoft Research tested what happens when frontier models make chained multi-step changes without human checks. Their DELEGATE-52 benchmark found 19-34% quality loss over 20 delegated iterations. Python workflows held up better, with less than 1% loss on average. That split is the key finding. Code holds its integrity where other formats do not. The authors flag this as a diagnostic stress test, not a capability measure. Production systems succeed precisely because of the verification loops the benchmark strips away.

🛠 2 Tools & Repos

1. fff

https://github.com/dmtrKovalenko/fff

fff is a file search tool for processes that run hundreds of queries per session, like AI agents and editors. Most search tools pay fork overhead on every call, which adds up fast. fff keeps an in-memory index resident instead. On a 500K-file Chromium checkout, that means sub-10ms results. The tool ranks results by how often and how recently you used them. It also handles typos in queries, respects gitignore, and classifies code definitions. Rust powers the core, with bindings for Neovim, C, and Node.js.

2. awesome-design-md

https://github.com/VoltAgent/awesome-design-md

Drop a DESIGN.md file into your project. Your AI coding agent builds pixel-accurate UI without Figma exports or manual design tokens. The repo includes over 70 ready-to-use design specs inspired by brands like Stripe, Apple, Spotify, and Figma. Each spec covers color palettes, typography, component styles, and spacing systems. The agent reads the markdown and generates consistent, branded interfaces.

🎓 1 Pick of the Week

Google Antigravity 2.0 + CLI + SDK

https://antigravity.google/blog/google-io-2026

Google released a full agent ecosystem at I/O 2026. Antigravity 2.0 is a standalone desktop app, not embedded in Chrome or Android. It runs on Gemini 3.5 Flash, which outperforms Gemini 3.1 Pro on Terminal-Bench 2.1 (76.2%). Inside Antigravity, an optimized version runs up to 12x faster than other frontier models (Google’s figure; its general headline number is 4x). The CLI brings agent workflows to the terminal. The SDK lets you build custom agents on the Antigravity harness. Managed agents deploy with a single Gemini API call. Multi-agent orchestration lets Gemini 3.5 Flash run many agents in parallel. The new $100-a-month Google AI Ultra plan gives 5x higher usage limits in Antigravity than the Pro plan.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my weekly resource roundup. I curate AI/ML papers, videos, articles, tools, and learning resources worth your time. Subscribe to Gradient Ascent for the full experience.

TL;DR

What caught my attention this week:

• 📄 Papers: Train a 120B beast on one H200 by parking weights in host RAM. One video model that covers every input type, no more one-model-per-modality. Four agents gang up on Anthropic’s GPU-kernel benchmark and beat it. A self-driving model that thinks without spelling it out, no accuracy hit. A training arena that grows its own environments.

• 🎥 Videos: Sebastian Raschka rebuilds modern LLMs layer by layer in 53 minutes. Anthropic teaches Claude to narrate Claude. Samuel Humeau makes the case that today’s TTS is basically an LLM with audio tokens. Dave Ebbelaar builds agentic RAG out of three tool primitives.

• 📰 Reads: OpenAI killed finetuning, and swyx says the rest of us are stuck on “just very long prompts.” Nathan Lambert on why open labs compound their R&D into a real cost advantage. Paul Iusztin’s six-agent Claude Code rig, spec to merged PR with two human gates.

• 🛠 Tools: Semantic code search as an MCP plugin for any coding agent. StepFun’s open audio reasoner that gave up on auto-scoring because the high scores were lying.

• 🎓 Learning: Matt Pocock open-sourced his personal `.claude` directory. Read it like a curriculum.

📄 5 Papers

1. MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

https://arxiv.org/abs/2604.05091 | GitHub

Train a 120B model on one H200. The authors park parameters and optimizer state in 1.5TB of host RAM, then stream them to the GPU layer by layer with a double-buffered handoff. At 14B they clock 1.84x faster than DeepSpeed ZeRO-3 with CPU offloading. One catch: if prefetch can’t keep up with compute, PCIe becomes the bottleneck.

2. UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors

https://arxiv.org/abs/2605.00658 | GitHub

Most video diffusion models are one-trick: one for plain video, another for video plus depth, another for video plus transparency. UniVidX is a single backbone that learns all of them. HKUST MMLab trained two flavors on under 1,000 videos each: video plus scene geometry, and video plus transparency. Both held their own against the specialists. They only tried two combinations, though. Whether the trick generalizes is still open.

3. CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery

https://arxiv.org/abs/2604.01658 | GitHub

Four LLM agents work in their own sandboxes. They take turns proposing solutions, critiquing each other’s work, and keeping what holds up in a shared memory. No fixed playbook tells them what to try next; the agents riff on each other’s ideas. On Anthropic’s GPU-kernel benchmark they cut the work from 1363 to 1103 cycles. Across 10 tasks in math, algorithms, and systems, they run 3 to 10x faster than fixed evolutionary baselines.

4. Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation

https://arxiv.org/abs/2604.18486 | GitHub

Chain-of-thought is too slow for self-driving. The model can’t write out its reasoning step by step in real time. So the authors compress that reasoning into a handful of hidden internal values the model processes in one shot, instead of one word at a time. Across four driving benchmarks, it’s the first compressed approach to match the speed of skipping reasoning entirely, without losing accuracy. The price: a three-stage training pipeline over trajectory, language, and vision objectives.

5. Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence

https://arxiv.org/abs/2604.18292

Most agent training uses a fixed task set. Agent-World grows the set on the fly: new tasks come from topic-specific knowledge bases, the agent gets realistic MCP-style tools, and they aim reinforcement learning at whatever the agent currently fails at. Tasks are “verifiable,” meaning a checker can score them automatically without a human in the loop. Their 8B and 14B models beat strong commercial baselines across 23 agent benchmarks. The abstract doesn’t say by how much. Only a project page is live so far. File under promising, show me the receipts.

🎥 4 Videos

1. Sebastian Raschka on What He Learned From Implementing Modern LLM Architectures From Scratch

Port open-weight LLMs into your own Python from scratch in 53 minutes. The trick: load HuggingFace weights, then verify layer by layer against the HF Transformers reference using random weights on a small block. Mismatches surface before you’ve wired the whole stack. Sebastian Raschka shows that recent architectures mostly differ in attention variants and sizing, driven by the rising memory cost of longer reasoning and agent traces. He closes with a beginner-to-expert roadmap of open-source projects.

2. Anthropic on Translating Claude’s Internal Activations Into English

A second Claude reads the first Claude’s mid-layer activations and emits English. Anthropic verifies fidelity by translating that English back into numbers and matching the original. In a simulated blackmail scenario, the translator catches Claude noticing the obvious: this looks like a safety evaluation.

3. Samuel Humeau (Mistral) on Why TTS Models Now Look Like LLMs

Modern TTS works like an autoregressive LLM, just predicting audio tokens instead of words. Samuel Humeau (Mistral) walks through why in 22 minutes. Audio tokenizes at 80ms frames, 12 per second, roughly 500 tokens per second of speech, against about 15 bits per second of actual semantic content. (Brutal ratio.) He demos the streaming trick with a small voice agent: emit packets before computation finishes. Mistral’s open release ships the decoder and voices, but no cloning encoder.

4. Dave Ebbelaar on Building Agentic RAG From Scratch in Pure Python

Dave Ebbelaar wires three simple tools and plain Python into an agent that searches its own knowledge base. In 27 minutes, he shows where the simpler approach still wins (fetch the relevant chunks once, answer), and where letting the agent run multi-step searches is worth it.

📰 3 Curated Reads

1. The End of Finetuning

Latent.Space

[AINews] The End of Finetuning

The proximal cause of today’s op-ed is OpenAI’s deprecation of their finetuning APIs…

Read more

2 months ago · 48 likes

OpenAI killing its finetuning APIs ends finetuning for the modal 80% of AI engineering. swyx ties this to OpenAI’s broader 2026 pullback (Sora cut, Anthropic about to pass them in valuation), while Cursor and Cognition, fresh off a $25B round, lean harder on reinforcement learning over open models. He traces the call back to Jeremy Howard on the Latent Space pod, October 2023. The middle of the market gets “Just Very Long Prompts.” Said with affection, presumably.

2. How Open Model Ecosystems Compound

Interconnects AI

How open model ecosystems compound

Note: Voice-overs for paywalled posts are available for paid subscribes in podcast apps if you click on settings on Interconnects, then manage your description. Thanks for listening…

Read more

2 months ago · 36 likes · 1 comment · Nathan Lambert

Roughly 80% of frontier-model compute goes to R&D, not the final training run. Nathan Lambert grounds that figure in Ai2’s Olmo 3 writeup and an Epoch AI cost analysis. He argues China’s open peer-sharing turns this into a structural cost advantage: their labs can build longer than outsiders expect. He also draws a clean line between “open AI” and open-source software. Open AI is about sharing risky R&D, not ready-to-deploy code.

3. From Vibe Coding to a Real Engineering Team

Decoding AI Magazine

From Vibe Coding to a Real Engineering Team

I needed a TypeScript harness for my latest book code. It required a Terminal User Interface (TUI), an agent loop, tools, Model Context Protocol (MCP) support, skills, and slash commands. I will be honest with you. I first tried to vibe code this project…

Read more

2 months ago · 40 likes · 7 comments · Paul Iusztin

Six Claude Code agents with one rule: no agent both writes code and judges it. Paul Iusztin’s roster: a PM-architect on ADRs, a TDD engineer in raw shell, an adversarial tester, a diff-only PR reviewer, and a CI-watching on-call agent. The raw-shell choice is deliberate: LLMs have seen far more bash code than MCP wrappers during training. The `/night` orchestrator runs spec to merged PR with two human gates.

🛠 2 Tools & Repos

1. zilliztech/claude-context

https://github.com/zilliztech/claude-context

Semantic code search as an MCP plugin for most major coding agents. It indexes your codebase into Zilliz Cloud or self-hosted Milvus, then serves targeted retrievals back into the agent’s context. One `npx` install, an OpenAI key for embeddings, a Milvus endpoint. Zilliz built it, so yes, it funnels you toward their hosted vector DB.

2. stepfun-ai/Step-Audio-R1

https://github.com/stepfun-ai/Step-Audio-R1

StepFun shipped an open audio reasoner. R1.5 dropped automated metrics in favor of human feedback (RLHF). The reason: scoring high on the objective tests didn’t make the model sound natural in conversation. The repo ships vLLM inference, weights for R1 and R1.1, a Gradio demo, and three new evaluation benchmarks. Apache-2.0. Watch the Humeau video above first. Step-Audio-R1 is what that direction looks like with weights you can run today.

🎓 1 Pick of the Week

mattpocock/skills

https://github.com/mattpocock/skills

Matt Pocock open-sourced his personal `.claude` directory as a runnable skills repo. Headliners: `tdd`, `diagnose`, `grill-me` (relentless interview before coding), `caveman` (~75% token compression), `handoff` (conversation compaction). Install with `npx skills@latest add mattpocock/skills`. Each skill reads like a short essay on the engineering principle behind it.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

This week’s housekeeping:

What: PMs getting real leverage from AI have rebuilt their workflow around it, right from how they spec, how they research, and all the way to how they ship.

Avinash Mahalingam (Touch Infinity, ex-PM leader at Amazon) and I are running a free 60-minute live session on the AI-Powered PM OS.

Come and learn an end-to-end walkthrough of the setup we use in Claude Code: a vague Slack request becoming a spec, competitor scans in minutes instead of afternoons, engineering staying aligned without drowning in status updates.

What you’ll leave with:

• A working picture of what an AI-PM workflow looks like in practice

• The three places most PMs are quietly losing time to AI-fluent peers

• A repeatable structure for plugging AI into spec writing, customer research, and stakeholder updates

• Live Q&A with Avinash and me

Who it’s for: Product Managers, Directors of Product, VPs of Product, and founders who own product. If you’ve been told to “use AI more” but haven’t found a setup that saves you hours a week, come.

When: May 15, 12 noon ET.

Register for FREE

New here?

The Tokenizer is my resource-focused newsletter edition where I curate high-signal AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: A YAML specification language for agents you can read top to bottom (77.1% on SWE-Bench Verified). A robot foundation model runs closed-loop at 12.7Hz on a sub-$6K arm. Reasoning rewards reduce scalar reward hacking on image generation. Minimal-sufficient knowledge hints carry most of the RL signal on hard reasoning problems. A purpose-built agentic stack for browser games.

• 🎥 Videos: Ravi Mehta rebuilds a music app prototype live in 40 minutes. IKEA’s team builds enterprise knowledge by watching agents fail at real incidents. METR’s authors spend 113 minutes on the misreadings of their time-horizons chart. Stripe’s Owen Williams walks through Protodash, the internal prototyping studio their PMs and designers use day to day.

• 📰 Reads: Microsoft share the state of AI at work. The EvalEval coalition lays out the eval-cost problem: a single GAIA run costs $2,829, and evaluation compute now sits two orders of magnitude above training compute. DeepMind’s Decoupled DiLoCo trains a 12B model across four U.S. regions on 2-5 Gbps and keeps running when a region fails.

• 🛠 Tools: A CLAUDE.md that bakes Karpathy’s four anti-patterns into your coding agent. A RAG framework that handles PDFs with charts and equations as one pipeline rather than four parsers.

• 🎓 Learning: graphify turns a folder of code, papers, and screenshots into a graph you can query, with 71.5x fewer tokens per question than reading the raw files.

📄 5 Papers

1. AgentSPEX: An Agent SPecification and EXecution Language

https://arxiv.org/abs/2604.13346 | GitHub

Most agent codebases bury the control flow inside Python and let the LLM handle the rest at runtime. UIUC’s AgentSPEX moves the workflow into human-readable YAML: explicit branching, iteration, context handoffs, all in one file you can read top to bottom. It runs inside a Docker harness that checkpoints, replays, and ships with over fifty built-in tools. They report top scores on all seven benchmarks they tested, including 77.1% on SWE-Bench Verified and 100% on AIME 2025.

2. MolmoAct2: Action Reasoning Models for Real-world Deployment

https://arxiv.org/abs/2605.02881 | GitHub

12.7Hz on a sub-$6K robot arm. That’s MolmoAct2 from Allen AI: a Vision-Language-Action model built for real-world deployment, with explicit targets on robustness, latency, and predictability. Allen AI has released weights, training code, and three new datasets covering bimanual, low-cost SO-100, and DROID setups. Real-world zero-shot hits 87.1% on DROID.

3. RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time

https://arxiv.org/abs/2604.11626 | GitHub

Scalar reward models on image generation reduce rich human judgments to a single score and discard the reasoning behind them. RationalRewards trains an 8B VLM to write a multi-dimensional critique (text faithfulness, image faithfulness, physical quality, text rendering) before it produces a score. They train it with a hindsight-foresight framework called PARROT, no human-annotated rationales required. Bolt it onto FLUX.1-dev via RL and it lifts UniGenBench++ by 9.37 points. Run it as a generate-critique-refine loop at test time and it matches or beats RL fine-tuning for about 0.4 seconds of extra inference per image.

4. KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance

https://arxiv.org/abs/2604.12627

RLVR improves reasoning, but it stalls on hard problems where rewards are sparse. KnowRL’s claim: a small number of well-chosen knowledge hints during training carries most of the signal, and piling on more just adds noise. They call this the “critical-segment effect.” Their Constrained Subset Search picks 2.57 knowledge points per problem on average. A 1.5B Nemotron model trained this way gains 9.63 points (15.11 on AIME25), measured without inference-time hints.

5. OpenGame: Open Agentic Coding for Games

https://arxiv.org/abs/2604.18394 | GitHub

Game development sits at the intersection of creative design and intricate engineering: engines, real-time logic, asset pipelines, all orchestrated together. OpenGame is a purpose-built agentic stack for it: a six-phase workflow, two evolving “Game Skills” that grow from a meta-skeleton into genre-specific templates, and a debug protocol that accumulates verified fixes instead of relearning them. Running on Claude Sonnet 4.6, it scores 72.4 on Build Health and 65.1 on Intent Alignment across 150 browser-game tasks. Their custom GameCoder-27B variant matches larger proprietary stacks on the same benchmarks.

🎥 4 Videos

1. A Three-Layer Context Engineering System for Vibe-Coded Product Prototypes

Ravi Mehta separates a prototype into three context layers: functional, visual, and data. Treat each one separately and the prototype stops looking like every other AI demo. He rebuilds a music app live in 40 minutes with Peter Yang. Halfway through, he stands up a custom MCP server inside Claude Code so the data layer queries real data instead of mocked data. He closes on the harder PM questions: when a prototype is enough, when to stop and write production code, when a PRD still earns the time spent on it.

2. Demand-Driven Context: Building Enterprise Knowledge Bases by Watching Agents Fail

Top-down enterprise knowledge bases keep failing. Raj Navakoti’s argument from IKEA Digital: hand the agent real incident-response work, watch where it fails, and let those failures tell you which tribal knowledge needs writing down. Across 68 minutes he sorts knowledge into three colors (general, taught, tribal), walks through a live root-cause-analysis demo across 14 incidents and watches confidence climb as gaps get filled, and lands on why Markdown in GitHub plus a meta model beats yet another wiki. Pair it with the Stripe video below.

3. METR’s Time Horizons Graph and Why Its Authors Are the Most Cautious About Reading It

You’ve seen the chart: longest task a frontier model can complete at 50% reliability, plotted against release date, log-linear, doubling every few months. It anchors most of the current AI-timelines debate. Beth Barnes and David Rein, two of the people who built it, spend 113 minutes with Machine Learning Street Talk on the misreadings they keep seeing. They walk through reward hacking, construct validity, the ARC-AGI 1-to-2 collapse, the SWE-bench finding that half of passing PRs would not get merged in real codebases, and Beth’s horses-versus-bank-tellers framing for what labour displacement looks like in practice. If you’ve cited the chart in the last six months, watch this before you cite it again.

4. Inside Stripe’s Protodash: How Designers and PMs Prototype Without Writing Code

Stripe built an internal AI prototyping studio that screenshots its own work to check whether it built the right thing. Owen Williams walks through Protodash in 55 minutes: it started as a bundle of Cursor rules plus React components and grew into a browser-based studio with design review modes, variant testing, and self-testing prototypes running in dev boxes. He covers the architecture (Cursor rules teaching the Stripe design system, MCP integrations) and the “blurple slop” problem with generic AI design tools. PMs ended up using Protodash as much as designers did, which nobody at Stripe expected.

📰 3 Curated Reads

1. New Future of Work: AI is Driving Rapid Change, Uneven Benefits

https://www.microsoft.com/en-us/research/blog/new-future-of-work-ai-is-driving-rapid-change-uneven-benefits/

Microsoft Research’s annual “New Future of Work” report summarises five years of fieldwork on AI in the workplace. 38% of employed Germans now use AI at work, and users report saving 40–60 minutes a day. 40% of U.S. employees received “workslop” (unusable AI-generated content) in the past month, and employment for workers aged 22–25 in AI-exposed roles has declined 16% relative to less-exposed roles. AI-skill job postings are nearly twice as likely to also demand analytical thinking and resilience. Worth reading to understand the state of play in the workforce.

2. AI evals are becoming the new compute bottleneck

https://huggingface.co/blog/evaleval/eval-costs-bottleneck

A single GAIA run on a frontier model costs $2,829. The EvalEval coalition lines up numbers like that one and names the structural shift: in scientific ML, evaluation compute now sits two orders of magnitude above training compute. Two things follow. Academic groups hit budget walls before they hit technical ones. And external validation of frontier models concentrates inside the same labs building them, a structural problem the field has not addressed.

3. Decoupled DiLoCo: A new frontier for resilient, distributed AI training

https://deepmind.google/blog/decoupled-diloco/

Original DiLoCo solved bandwidth for distributed training. It didn’t fix the bigger problem: one bad node still stalls everyone else. Decoupled DiLoCo (Arthur Douillard’s team at Google DeepMind) splits training into independent compute “islands” so chip failures in one region don’t block the others. On a 1.2M-chip simulation, that’s 88% goodput under failure versus 27% for conventional data-parallel. A 12B model trained across four U.S. regions ran more than 20x faster than conventional sync on 2-5 Gbps of wide-area networking, with effectively no accuracy cost (64.1% vs 64.4% baseline).

🛠 2 Tools & Repos

1. forrestchang/andrej-karpathy-skills

https://github.com/forrestchang/andrej-karpathy-skills

Four rules in one CLAUDE.md file, derived from Andrej Karpathy’s public observations on how LLMs code: think before coding, simplicity first, surgical changes, goal-driven execution. Forrest Chang translated each into something an agent can act on. Each rule targets a specific failure mode: silent assumptions, bloated abstractions, drive-by edits to code you didn’t ask about, “make it work” tasks with no verification loop. Drop it in your repo root before the next session.

2. HKUDS/RAG-Anything

https://github.com/HKUDS/RAG-Anything

Most RAG stacks struggle with documents that mix modalities: a PDF with images, an Office file with tables, a paper with LaTeX equations, a chart embedded in prose. RAG-Anything from HKU’s data lab handles all of those as one pipeline. It parses with MinerU, routes each modality (text, image, table, equation) through a dedicated analyzer, builds a cross-modal knowledge graph, and runs hybrid vector-plus-graph retrieval so a query can hit a chart caption and the paragraph that explains it in the same call. The newer VLM-enhanced query mode pulls images straight into the vision model at query time. Pip install raganything, point it at a directory of mixed-format docs, and stop writing one-off parsers per file type.

🎓 1 Pick of the Week

safishamsi/graphify: Turn Any Folder Into a Queryable Knowledge Graph

https://github.com/safishamsi/graphify

Point graphify at a folder of code, papers, screenshots, and notes, and it builds a single graph you can query. `/graphify .` runs inside Claude Code: tree-sitter for source files, Claude vision for images and diagrams, and a Leiden-clustered NetworkX graph holding it all together. It reports god nodes, surprising cross-references, and (their measurement, on a mixed corpus of Karpathy repos plus papers plus images) 71.5x fewer tokens per query than reading the raw files. Good fit for any folder you’ve been quietly accumulating: agent skills, research bookmarks, screenshots, notes.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate high-signal AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: an organisational layer for agent teams that scores 84.67% on a product-spec benchmark, a way to make multi-agent systems “loop” the way single models do, a test-driven approach to building training data, a safety survey for robots that see-and-act, and a scaling law that prices one extra loop at 0.46 of a fresh parameter.

• 🎥 Videos: an OpenAI workshop on Codex with custom subagents, Dave Ebbelaar’s five-level taxonomy of agent complexity, a Stanford seminar on world models that predict in compressed space rather than pixel space, and Sasha Rush on building Cursor Composer.

• 📰 Reads: a source-code walk through Claude Code’s internals, Eugene Yan’s three-step workflow for evaluating AI products, and Tom Aarsen on a single library that now embeds text, images, audio, and video into one shared space.

• 🛠 Tools: Zilliz’s claude-context, which turns your codebase into searchable context for a coding agent, and trycua/cua, an open-source toolkit for letting agents drive macOS, Linux, Windows, and Android.

• 🎓 Learning: HuggingFace’s ml-intern, an open-source ML engineer agent that reads papers, pulls datasets, runs training jobs, and iterates on its own evaluations.

  ---

📄 5 Papers

1. From Skills to Talent: Organising Heterogeneous Agents as a Real-World Company

https://arxiv.org/abs/2604.22446 | GitHub

Most multi-agent papers try to fix the individual agent. This one argues the bottleneck is one level up: there is no org chart. Nothing decides who works on what, who reports to whom, or how the team improves over time. OneManCompany (OMC) packages skills, tools, and configs into reusable workers it calls “Talents,” lets agents recruit them on demand from a shared pool, and runs the team through a plan-act-review loop that is mathematically guaranteed not to get stuck. On PRDBench (which tests whether agents can write production-quality product requirement docs), OMC scores 84.67%, beating the prior best by 15.48 percentage points. If you’ve spent the last six months wiring up bespoke crews per task, this is the paper that names the thing you’ve been working around.

2. Recursive Multi-Agent Systems

https://arxiv.org/abs/2604.25917 | GitHub

Some language models get smarter by running the same internal layers multiple times over a problem, refining a “thought” each pass. RecursiveMAS lifts that trick to teams of agents: instead of negotiating in plain text, agents hand each other a compressed thought and refine it round by round through small connector modules. The whole system is trained end-to-end so the team learns where each round contributed. Across nine benchmarks spanning math, science, medicine, search, and code, the framework reports an average 8.3% accuracy gain while using up to three-quarters fewer tokens. Pair it with Iso-Depth (paper #5): looping is a real axis to scale on, not a curiosity.

3. Programming with Data: Test-Driven Data Engineering for Self-Improving LLMs (ProDa)

https://arxiv.org/abs/2604.24819 | GitHub

I found this analogy quite sticky and memorable. Training data is your source code. Training is the compile step. Benchmarks are your unit tests. Failure analysis is debugging. When the model fails, the failure traces back through the data the way a stack trace traces back through a function call. You patch the dataset where the gap actually lives, not the model. The team instantiates the loop across sixteen disciplines covering natural sciences, engineering, biomedicine, and social sciences, and ships a structured knowledge base, benchmark suite, and training corpus alongside the 57-page paper. If your fine-tuning pipeline is “train, eval, scratch head, add more data,” this is the pattern that retires it.

4. Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms

https://arxiv.org/abs/2604.23775 | GitHub

A reference for anyone building or auditing robots and agents that see, read, and then act in the world (the field calls them Vision-Language-Action models, or VLAs). The survey maps the threat surface across two axes: when the attack happens (during training vs while the model is running) and which side you are on (attacker vs defender). It walks through hidden triggers planted in training data, jailbreaks that exploit how the model reads a scene, visual noise designed to fool it, and physical interventions in the real world. The defence half covers cleaner data, better reward signals, policy-level guardrails, runtime monitors, and physical fail-safes. The companion repo lists roughly 70 papers and resources with links. Pull this the next time someone hands you a VLA demo and asks if it’s safe to ship.

5. How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped LMs

https://arxiv.org/abs/2604.21106 | GitHub

How much is one extra loop through a model’s own layers worth? About 0.46 of a fresh layer. Schwethelm, Rueckert, and Kaissis settle the argument with 116 training runs that vary the loop count (1, 2, 4, or 8) across roughly 50× of training compute, total depth pinned at 20. The endpoints anchor the scale: a fully shared loop would be worth 0%, a fully fresh layer 100%. A 410-million-parameter looped model matches a 580-million-parameter non-looped model on validation loss, though it costs about as much to train as a 1-billion-parameter one. Read this before your next architecture argument about whether weight-sharing is “free.”

🎥 4 Videos

1. Codex and Subagents: A 62-Minute Workshop

Vaibhav Srivastav and Katia Gil Guzman walk through the current Codex stack: the underlying models (GPT-5.3, Spark, GPT-5.4), the app and command line, plugins, automations, and custom subagents that each get their own model, permissions, and tools. The middle third is a live demo that runs from a Google Drive plugin to a Slack and Gmail automation to a GitHub code review to subagents reviewing persona files. The closing minutes touch the newer surface area: approval gates, hooks, personality settings, and the Cloud Code plugin. Sits next to ml-intern (this week’s Pick) and claude-context below: same AI-workforce idea, three surfaces.

2. Five Levels of AI Agent Complexity, From Augmented LLM to Multi-Agent

Dave Ebbelaar’s 22-minute breakdown is the clean taxonomy you can hand to a non-engineer when they ask “what is an agent.” Level 1 is an LLM with extra context. Level 2 is chaining and routing prompts in fixed steps. Level 3 is letting the model call tools. Level 4 is wrapping the model in a harness that decides what runs next. Level 5 is multiple agents coordinating. Code is on GitHub (using two common Python libraries for building agents), with traces from a live client system. The honest part is the framing of when each level is enough and when it is not. Worth 22 minutes if you are about to over-architect a workflow.

3. Stanford CS25 V6: From Representation Learning to World Modeling

World models used to mean: predict every pixel of the next frame. The change that Hazel Nam and Lucas Maes (Brown) walk through in this 71-minute Stanford seminar is to predict in a compressed, abstract space instead: fewer pixels, more meaning. This is the shortest route to where world-model research is in early 2026 without spending four hours on arxiv.

4. Sasha Rush on Building Cursor Composer at Ray Summit 2025

A 19-minute Anyscale keynote from Sasha Rush on the design behind Cursor Composer: how the agent reasons across a codebase, how speed and intelligence trade off when you ship to working developers, and what it took to make the experience feel uninterrupted. The model details and the product surface get equal weight, which is what you want when the agent is the product. Pair it with the Inside Claude Code read below for two takes on the same design problem.

📰 3 Curated Reads

1. Inside Claude Code: An Architecture Deep Dive

https://zainhas.github.io/blog/2026/inside-claude-code-architecture/

Zain Hasan walks the leaked Claude Code source (1,884 files, roughly 512,000 lines) across eleven sections, with ASCII diagrams and narrative pairs. The four-layer stack, the main query loop, the Tool interface, the permission cascade, and a deep dive on how the Grep tool wraps the ripgrep search utility are all useful. How Claude Code manages memory is the standout for me: five layers in all, including budgets for how much each tool result can return, automatic snipping of older content, micro-edits that update the cache in place, model-side summarisation, and a final auto-compaction that kicks in when the conversation gets within 13,000 tokens of the limit. Note that the author flags upfront that the post is a live document written with Claude’s help while reading the repo, which is itself a demo of the workflow it describes.

2. Product Evals in Three Simple Steps

https://eugeneyan.com/writing/product-evals/

Eugene Yan lays out a workflow you can run on Monday. Step one: pull roughly 200 real samples from production and hand-label 50 to 100 failure cases. Step two: train an LLM to judge new outputs the way you would. Use a binary pass/fail rather than a 1-to-5 scale (the gap between a 3 and a 4 is too hard to pin down), and check the judge against your labels using precision, recall, and an agreement score above 0.7. Step three: wire the whole eval loop into your experiment pipeline so every change produces a verdict without anyone running anything by hand. He splits the labelled samples 75% for development, 25% as a held-out test. Bookmark it the next time someone on your team says “we should add evals.”

3. Multimodal Embedding and Reranker Models with Sentence Transformers

https://huggingface.co/blog/multimodal-sentence-transformers

Tom Aarsen at HuggingFace walks through the new release of Sentence Transformers, the popular open library for turning text into searchable vectors. The release adds one shared interface that now embeds text, images, audio, and video into the same space. One model load, one `encode()` call across all four, and cross-type similarity that just works (a “green car” text query scores 0.51 against a car image and 0.10 against a bee image). The reranker scores mixed-type pairs the same way. The retrieve-and-rerank pattern at the end is exactly the pipeline most teams ship: a fast first pass to pull the top candidates, then a smaller, slower model to reorder them for accuracy.

🛠 2 Tools & Repos

1. zilliztech/claude-context

https://github.com/zilliztech/claude-context

This is an MCP plug-in from the Zilliz team that turns your codebase into searchable context for a coding agent. It splits code into chunks based on the actual code structure, re-indexes only the files that changed since last time, and runs a hybrid search that combines classic keyword matching with vector similarity (the same combo that powers most modern search engines). They report roughly 40% fewer tokens compared to dumping whole directories into context. If you have been hand-feeding your coding agent files, this might be the install that ends that habit.

2. trycua/cua

https://github.com/trycua/cua

Cua ships four pieces: a driver that automates native macOS apps in the background without hijacking your cursor, a sandbox that gives agents one common interface to virtual machines and containers across macOS, Linux, Windows, and Android, a desktop sandbox command line for coding agents with video streaming and a shared clipboard, and a benchmark suite for measuring how well an agent actually performs on common desktop tasks. If you are building a computer-use agent and want infrastructure you can actually inspect, this is where to start.

🎓 1 Pick of the Week

huggingface/ml-intern: An Installable ML Engineer Agent

https://github.com/huggingface/ml-intern

ml-intern is an agent that “autonomously researches, writes, and ships good quality ML related code using the Hugging Face ecosystem.” It reads papers, follows citations, pulls datasets from the HuggingFace Hub, runs training jobs, and iterates on its own evaluations. Two ways to run it: type `ml-intern` for an interactive session, or `ml-intern “your prompt”` for a hands-off run that auto-approves its own steps. Works with Anthropic or OpenAI behind it; needs HuggingFace and GitHub API keys. Hand it a paper you have been meaning to reproduce and see what the next morning looks like.

That’s the twenty-sixth Tokenizer. The interesting question for next week: which of these you actually install on Friday, and whether Monday’s repo looks different because of it. The long-form work continues at Gradient Ascent.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate high-signal AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: a source-level dissection of Claude Code, a self-evolving agent that argues for information density over context length, ByteDance’s next-generation video model, Tencent Hunyuan’s unified 3D world generator, and a principled recipe for on-policy distillation.

• 🎥 Videos: the physics behind Flow Matching, running LLMs locally on DGX Spark, engineering RL environments from scratch, and an anonymous operator conversation on AI budgets and code review.

• 📰 Reads: Hamel Husain on why eval craft belongs to data scientists, Paul Iusztin on harness engineering as the new OS layer around the LLM, and a transcript from inside OpenAI Frontier where a 1M-LOC codebase ships at a billion tokens per day with no human-written code.

• 🛠 Tools: Microsoft’s new agent framework with Semantic Kernel and AutoGen migration paths, and a Karpathy-style autoresearch loop for GPU kernels.

• 🎓 Learning: BMAD-METHOD, an open agile framework with six named agents that scaffolds AI-driven teams from brief to deployment.

📄 5 Papers

1. Dive into Claude Code: The Design Space of Today’s and Future AI Agent Systems

https://arxiv.org/abs/2604.14228 | GitHub

VILA Lab walked Claude Code’s TypeScript source (v2.1.88) and found that roughly 1.6% of the codebase is AI decision logic. The other 98.4% is operational infrastructure: permissions, context compaction, hooks, subagent isolation, session persistence. They extract five human values and thirteen design principles from that code, then trace them through seven components and five layers using a single running example (fixing a failing test in auth.test.ts). Read it if you’re building an agent and want to see how a production harness answers every design question at once.

2. GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)

https://arxiv.org/abs/2604.17091 | GitHub

Long-horizon agent performance is bounded by information density inside a finite context window, not by context length itself. GenericAgent builds on four pieces. A minimal atomic tool set. Hierarchical on-demand memory. A self-evolution loop that turns verified trajectories into reusable SOPs and code. A runtime compression layer. The repo reports 188K tokens on long-horizon tasks where Claude Code spends 537K, at equal or better completion rates. Read this before you reach for the next context-length upgrade.

3. Seedance 2.0: Advancing Video Generation for World Complexity

https://arxiv.org/abs/2604.14148

ByteDance Seed’s latest video model takes the #1 spot on both Text-to-Video and Image-to-Video leaderboards on Arena.AI, ahead of veo-3.1-audio-1080p on T2V and grok-imagine-video-720p on I2V. On the team’s own SeedVideoBench 2.0 set (against Kling, Sora, Veo, Wan, and Vidu), it ranks first in 29 of 30 fine-grained motion quality categories. The architectural bet is unified audio-video joint generation with native multi-modal input (text, image, audio, video) and simultaneous multi-track audio output with binaural synthesis. Pull the report if you want to see the current ceiling in open-access video generation.

4. HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

https://arxiv.org/abs/2604.14268 | GitHub

A single pipeline that does both 3D world generation and reconstruction, producing navigable 3D Gaussian Splatting scenes from text or one image alone. Tencent Hunyuan’s four-stage approach covers panorama generation, trajectory planning, camera-guided view generation, and final 3D composition. It reaches state-of-the-art among open-source 3D world models and reports parity with the closed Marble system. All weights and code are public, which matters for a field where most of the good stuff is still locked behind APIs.

5. Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe

https://arxiv.org/abs/2604.13016 | GitHub

On-policy distillation keeps breaking in counterintuitive ways: a stronger teacher sometimes fails to improve a student while a weaker one succeeds. THUNLP’s team runs controlled experiments across the Qwen3 and DeepSeek families and lands on two governing conditions. Student and teacher need compatible thinking patterns (measurable via token-level overlap ratio), and the teacher needs to offer capabilities the student doesn’t already have (not just a higher benchmark score). They also propose an off-policy cold-start warm-up that recovers distillation in setups where it would otherwise collapse. If your post-training pipeline picks teachers by benchmark score, the overlap-ratio diagnostic here is the first thing to add.

🎥 4 Videos

1. The Physics Behind Flow Matching, Derived from Scratch

Flow Matching built up from the continuity equation and time-variant velocity fields, with the training loss arriving only after continuous normalizing flows are in hand. Julia Turc’s 24-minute walkthrough pairs with a free interactive tutorial at diffusion.fyi. It covers conditional velocity fields in the spot where most explanations either gloss over optimal transport or collapse into notation. Queue this when your current mental model of flow matching has a gap you cannot quite name.

2. Running LLMs Locally: Practical Performance on NVIDIA’s DGX Spark

Serving open-source models from 1.5B to 14B parameters on a single DGX Spark with vLLM, with the throughput, latency, and quantization trade-offs that drop out once you measure them. Mozhgan Kabiri Chimeh from NVIDIA shares a reproducible methodology, NVFP4 performance numbers on Grace Blackwell’s 128GB unified memory, and a framework for local model sizing. A 10-minute talk if you’re deciding whether on-prem compute clears the bar for your workload.

3. Engineering Reinforcement Learning Environments Like Software

RL environments treated as first-class software artifacts, not research scaffolding. Stefano Fiorucci, AI engineer at Deepset, uses the open-source Verifiers library to translate classical RL concepts to language models. He builds single-turn tasks, multi-turn games, and tool-using agents as concrete environments. Watch it before you write your next eval harness. A clean gym spec will save you rebuilding it three weeks in.

4. Stay Sassy and swyx on AI Budgets, Per-Person Token Spend, and Why Code Review Matters More

Per-person token budgets are a management problem, not a provisioning one, and code review gets more important as agents write more code, not less. That is the spine of a 57-minute Latent Space conversation between swyx and Stay Sassy, an anonymous operator with voice modulated for opsec. They cover build-vs-buy, where hand-coding still wins, and how real engineering leaders are allocating AI spend. Closest thing to eavesdropping on the hard numbers right now.

📰 3 Curated Reads

1. The Revenge of the Data Scientist

https://hamel.dev/blog/posts/revenge/

Training a model was never most of a data scientist’s job. The bulk of the work was running experiments to test how well a system generalizes, debugging stochastic behaviour, and designing metrics you can actually trust. Hamel Husain argues that LLM teams who cut data scientists out of the loop are now rediscovering the same five eval pitfalls, from off-the-shelf judge metrics that flatter bad systems to dashboards nobody would ship a classifier against. The piece names the concrete moves (error analysis from real traces, aligning LLM judges against human labels, constructing trust-metrics you would actually stake a launch on) and is the cleanest argument this year for why the old craft is now the new infrastructure.

2. Agentic Harness Engineering

Decoding AI Magazine

Agentic Harness Engineering

At the AI start-up I’ve been working at, building a financial personal assistant, we implemented LlamaIndex, added the Model Context Protocol (MCP), and built complex Retrieval-Augmented Generation (RAG) pipelines. Each piece added complexity without adding direct business value…

Read more

4 months ago · 78 likes · 12 comments · Paul Iusztin

The harness is where production agents actually live: tools, memory, sandboxes, and orchestration that let an LLM recover from failures, bridge context windows, serve multiple interfaces, and hold state across sessions. Paul Iusztin dissects the production harness behind Claude Code and Codex across a handful of components (context engineering, memory, sandboxing, tool layers, orchestration). He walks through how each piece solves a problem the model cannot solve on its own. The thesis: strip RAG pipelines and MCP layers back to plain Python until the harness itself earns the complexity.

3. Extreme Harness Engineering for Token Billionaires

Latent.Space

Extreme Harness Engineering for Token Billionaires: 1M LOC, 1B toks/day, 0% human code, 0% human review — Ryan Lopopolo, OpenAI Frontier & Symphony

We’re proud to release this ahead of Ryan’s keynote at AIE Europe. Hit the bell, get notified when it is live! Attendees: come prepped for Ryan’s AMA with Vibhu after…

Listen now

3 months ago · 45 likes · 4 comments

OpenAI’s Frontier team wrote a 1M-line codebase at 1 billion tokens per day with zero human-written code and zero human-reviewed merges. Ryan Lopopolo walks through how they got there. The build system went from Make to Bazel to Turbo to Nx to hit a sub-one-minute loop. Observability was exposed to the agent so it can tell when it is going off track. Dependencies were inlined to remove version drift. Specs were written for the model, not the engineer. The conversation with Swyx and Alessio covers the agent code-review rules, autonomous merging, and why human attention is now the binding constraint. Read it before you argue with anyone about what a production agent harness should look like in 2026.

🛠 2 Tools & Repos

1. Microsoft Agent Framework

https://github.com/microsoft/agent-framework

Microsoft’s unified framework for building and orchestrating AI agents across Python and .NET, with graph-based workflows, streaming, checkpointing, human-in-the-loop, and time-travel. The interesting move is that it ships with explicit migration guides from both Semantic Kernel and AutoGen, positioning itself as the convergence point for Microsoft’s prior agent SDKs. 9,736 stars and active weekly office hours. If you already run Semantic Kernel or AutoGen in production, the migration guides are the fastest read on where Microsoft wants your next agent to live.

2. AutoKernel: Autoresearch for GPU Kernels

https://github.com/RightNow-AI/autokernel

Karpathy-inspired autonomous loop that takes any PyTorch model, profiles it, extracts bottleneck kernels into standalone Triton or CUDA C++, and optimizes them one at a time in a keep-or-revert loop. Each experiment runs in about 90 seconds, roughly 40 per hour, 320 overnight. The orchestrator picks the next kernel using Amdahl’s law. 1,295 stars. Point it at a model you care about before bed and wake up to a speedup report.

🎓 1 Pick of the Week

BMAD-METHOD: Breakthrough Method for Agile AI Driven Development

https://github.com/bmad-code-org/BMAD-METHOD

BMAD-METHOD is an MIT-licensed framework that installs a full agile team of named AI agents into your IDE. You get six roles: Mary (business analyst), John (PM), Winston (architect), Sally (UX), Amelia (senior engineer), and Paige (tech writer). Each carries a consistent persona, a defined skill set, and a handoff point to the next. The core module ships 34+ workflows across brainstorming, PRD drafting, architecture, sprint planning, and code review. A solo developer can run a structured delivery loop without a real team. Install is one command (`npx bmad-method install`), and the framework picks up Claude Code, Cursor, and other IDEs on first run. 45K+ stars, actively maintained. Pick it up if you have been reaching for individu coding agents and find the scaffolding thinner than the model itself.

That’s the twenty-fifth Tokenizer. If one of these fifteen resources changes what you’re building this week, forward the edition to whoever should see it next, and come find the long-form work at Gradient Ascent.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate high-signal AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: a cleaner RL gradient signal that doesn’t collapse late in training, the GUI agent stack done end-to-end for once, memory-aware reward shaping that notices recurrent failure modes, a four-frame streaming video baseline that quietly beats thirteen heavyweight ones, and retrieval supervision pulled straight from agent trajectories.

• 🎥 Videos: Notion on custom-agent evals at scale, the Gemma 4 architecture delta, Claude Cowork for non-engineers, and a recipe for LLM judges that don’t silently drift.

• 📰 Reads: Agent swarms writing faster CUDA kernels at Cursor, a hands-on multimodal embedding tutorial from Hugging Face, and a clean first-principles tour of mathematical modeling.

• 🛠 Tools: VoxCPM2, a tokenizer-free multilingual TTS system, and a viral Claude/Codex plugin that cuts output tokens ~75% by making the agent talk like a caveman.

• 🎓 Learning: Stanford’s Spring 2026 Frontier Systems course, weekly lectures from the people building the AI infrastructure stack.

  ---

📄 5 Papers

1. Self-Distilled RLVR

https://arxiv.org/abs/2604.03128

On-policy self-distillation (OPSD) was supposed to give RL training a denser gradient signal. In practice it leaks privileged teacher information into the student and destabilizes training past the early-peak stage. RLSD keeps RLVR’s environmental feedback as the direction signal and uses self-distillation only to set token-level update magnitudes. The result: +4.69% average over the base LLM and +2.32% over GRPO across five multimodal reasoning benchmarks (MMMU, MathVista, MathVision, ZeroBench, WeMath), without the late-stage collapse OPSD exhibits.

2. ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents

https://arxiv.org/abs/2604.11784

GUI agents aren’t stuck because the models are weak. They’re stuck because the training pipelines, eval harnesses, and deployment layers keep breaking in public, and nobody’s addressed all three at once. The authors cover all three layers: RL training with GiGPO plus a process reward model, 95.8% evaluation reproduction across 6 benchmarks, and cross-platform deployment to Android, HarmonyOS, and iOS with hybrid CLI-GUI control. ClawGUI-2B trained inside this pipeline hits 17.1% success on MobileWorld GUI-Only, beating the same-scale MAI-UI-2B baseline by 6 points and larger untrained models like UI-Venus-72B.

3. The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping

https://arxiv.org/abs/2604.11297

RL for LLMs keeps collapsing into the same wrong answers. Entropy regularization penalizes randomness within the current policy but ignores recurrent failure patterns across rollouts. MEDS stores historical model representations, clusters them with HDBSCAN to surface common error modes, and down-weights reward for rollouts landing in high-density error clusters. Up to 4.13 pass@1 points gained on math reasoning benchmarks, with measurably higher behavioral diversity and negligible compute overhead.

4. A Simple Baseline for Streaming Video Understanding

https://arxiv.org/abs/2604.02317

Streaming-video models keep stacking heavier memory mechanisms. A sliding window of 4 recent frames into an off-the-shelf VLM (SimpleStream) matches or beats 13 specialized baselines: 67.7% on OVO-Bench and 80.59% on StreamingBench. The deeper finding is a perception-memory trade-off. Longer context often helps recall but hurts real-time perception. Future streaming benchmarks need to separate the two or they’ll keep rewarding machinery for its own sake.

5. LRAT: Learning to Retrieve from Agent Trajectories

https://arxiv.org/abs/2604.04949 | GitHub

Retrieval models trained on human click and dwell logs are mismatched to how LLM agents query and consume results. LRAT derives retrieval supervision directly from agent trajectories (browses, unbrowsed rejections, and post-browse reasoning traces) with weighted relevance intensity. Evidence recall, task success, and execution efficiency all improve across agent scales, and the BM25 and FAISS pipelines are in the repo.

🎥 4 Videos

1. Notion’s Head of AI Engineering on Running Custom-Agent Evals at Scale

When every Notion user has a bespoke agent, eval becomes a combinatorial problem. Sarah Sachs (head of AI engineering at Notion) and co-founder Simon Last walk Latent Space through how they keep the signal alive through that explosion. Technical interview, not product marketing. Worth it if you’re past the “does the agent work on my test case” stage and need patterns for eval at scale.

2. What’s New in Gemma 4

A 60-second official overview from Google DeepMind on what changed between Gemma 3 and Gemma 4. Treat it as a trailer, not a tech report. The architecture and training deltas it calls out are the right ones to chase down in the model card afterward. Worth including because the open-weights frontier is where half the interesting work this edition lives.

3. Claude Cowork Tutorial for Non-Engineers with JJ Englert (Tenex)

First hands-on Claude Cowork walkthrough aimed at people without an engineering background. JJ Englert runs end-to-end setup and a real workflow, which is the kind of demo that travels because it proves the tool works without scaffolding. Watch this for the first clean demo of Cowork running end-to-end without an engineer in the loop.

4. Judge the Judge: Building LLM Evaluators That Actually Work with GEPA

Mahmoud Mabrouk from Agenta AI at AIE Europe on building LLM judges that don’t silently drift. GEPA (Gradient-free Evaluator Prompt Adaptation) is a practical recipe for keeping your judge calibrated as the underlying model and task distribution shift. Directly useful if you’re running eval harnesses in production and can’t afford quiet regressions.

📰 3 Curated Reads

1. Speeding Up GPU Kernels by 38% With a Multi-Agent System

https://www.cursor.com/blog/multi-agent-kernels

Cursor and NVIDIA show a coordinated agent swarm writing CUDA kernels (CUDA C with inline PTX, CuTe DSL, a shared markdown scratchpad as the coordination medium) that hits a 38% geomean speedup across 235 real kernels. Standout wins include 84% on grouped-query attention and real gains on MoE GEMMs where human-tuned baselines were already aggressive. Read this for the concrete coordination pattern, not the headline number alone.

2. Multimodal Embedding and Reranker Models With Sentence Transformers

https://huggingface.co/blog/multimodal-sentence-transformers

Tom Aarsen’s step-by-step guide to building cross-modal retrieve-and-rerank over text, image, audio, and video using Qwen3-VL-Embedding and Reranker. Working code you can copy tonight, with the full retrieve-then-rerank pipeline wired up in Sentence Transformers. Rare to get the full modern multimodal RAG stack wired up in one notebook. Clone it, run it, then decide whether you need anything more than this.

3. The Power of Mathematical Modeling

The Palindrome

The Power of Mathematical Modeling

Hey! It’s Tivadar from The Palindrome…

Read more

3 months ago · 43 likes · 3 comments · Tivadar Danka and Manlio De Domenico, Ph.D.

Manlio De Domenico guest-posts on Tivadar Danka’s Palindrome with a first-principles tour of mathematical modeling: the SI compartment model applied to the ILOVEYOU worm, a four-compartment SIZR extension for zombie outbreaks, and a network-science result showing that immunizing hubs beats random interventions. The SI-to-SIZR extension is the kind of move that sticks. You watch a textbook model flex to handle zombies without losing any of its explanatory structure, which is exactly the feel you want from a modeling tour. The hub-immunization result at the end is the payoff. Drop this into the week when the systems-and-tooling fatigue catches up with you.

🛠 2 Tools & Repos

1. VoxCPM2: Tokenizer-Free Multilingual TTS

https://github.com/OpenBMB/VoxCPM

Tokenizer-free TTS system that generates continuous speech directly through a diffusion autoregressive architecture, bypassing discrete audio tokenization entirely. 30 languages, controllable voice cloning, voice design from text descriptions, and 48kHz studio-quality output. Over 13k stars and climbing fast. Pick this up if you’re building anything speech-facing and want to skip the usual codec-and-vocoder tax.

2. caveman

https://github.com/JuliusBrussee/caveman

A Claude Code / Codex / Gemini CLI plugin that instructs the agent to drop articles, filler, and pleasantries in its output while preserving technical content. Benchmarks in the README clock ~75% output token reduction and ~46% input token reduction with no quality loss on downstream evals. 30,800 stars in 11 days since launch. MIT, active maintainers, one-line install across seven agents. Worth the three-minute install if you’ve watched an agent burn context on “Certainly! I’d be happy to help you with that.” Note: Try it and test your luck :)

🎓 1 Pick of the Week

Stanford CS 153: Frontier Systems (Spring 2026)

https://www.youtube.com/playlist?list=PL2aDf5-VARtBwz1kz5FsuSZXOig2U6aJI

Stanford’s new Spring 2026 course on the AI infrastructure stack, taught by Anjney Midha (AMP PBC) and Michael Abbott. Weekly lectures from the people building the frontier: Andreas Blattmann on Black Forest Labs’ image models, Mati Staniszewski on ElevenLabs’ audio stack, and Midha himself on the infrastructure rewrite underneath this moment. Runs through June 3, with upcoming guests including Karpathy, Jensen Huang, Sam Altman, and Satya Nadella. The closest thing to a live running commentary on the stack as it’s being built.

That’s the twenty-fourth Tokenizer. If one of these fifteen resources changes what you’re building this week, forward the edition to whoever should see it next, and come find the long-form work at Gradient Ascent.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

TL;DR

What caught my attention this week:

• 📄 Papers: Benchmarks fight back against saturation, with new evaluations for video understanding, autonomous agents, robot policies, and the data behind LLM training.

• 🎥 Videos: Inside views from Stanford, OpenAI, Galileo, and Databricks on how agents actually run when you put them in front of real engineers (and non-engineers).

• 📰 Reads: Three sharp takes on coding agent architecture, why your RAG pipeline is overbuilt, and what really decides whether an open model gets adopted.

• 🛠 Tools: Memory and a Rust-native agent loop, two pieces of the open agent stack worth knowing.

• 🎓 Learning: A reproducible walkthrough of using Claude Code as a personal operating system, not as a coding tool.

📄 5 Papers

1. Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding

https://arxiv.org/abs/2604.05015 | GitHub

Video benchmarks have saturated to the point where leaderboard scores tell you almost nothing about real model capability. Video-MME-v2 introduces a tri-level hierarchy that escalates from visual aggregation up through temporal reasoning, plus a group-based scoring rule that punishes lucky guesses across linked questions. The team logged about 3,300 human-hours across 12 annotators and 50 reviewers, and one finding stands out: models lean hard on subtitles, and reasoning quality drops sharply when only the pixels are available.

2. DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

https://arxiv.org/abs/2603.26164

If you’ve ever tried to compare data selection, mixture optimization, and reweighting research, you know each lives in its own incompatible codebase. DataFlex sits on top of LLaMA-Factory and gives you one set of trainer abstractions for sample selection, DoReMi/ODM-style mixture tuning, and reweighting, all DeepSpeed ZeRO-3 compatible. It’s an infrastructure contribution rather than a new method, so the value here is reproducibility and composability across data-centric techniques you already wanted to try.

3. Adam’s Law: Textual Frequency Law on Large Language Models

https://arxiv.org/abs/2604.02176 | GitHub

The headline claim sounds tautological (frequent text is better text), but the paper actually proposes a measurement framework with three concrete components: a Textual Frequency Law that estimates sentence-level frequency from open web sources, a distillation step (TFD) that refines that estimate by querying the target model itself, and a curriculum (CTFT) that orders training data from rare to frequent expressions during fine-tuning. Tests cover math reasoning, machine translation, commonsense, and tool calling, which is a wider footprint than the abstract suggests.

4. Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents

https://arxiv.org/abs/2604.06132 | GitHub

Most agent benchmarks only check the final answer, which means a trajectory full of safety violations can still score a clean pass. Claw-Eval grades the path, not just the destination, with 2,159 fine-grained rubric items across 300 tasks, scored over execution traces, audit logs, and environment snapshots. The headline result is that trajectory-opaque grading misses 44% of safety violations and 13% of robustness failures, which is a hard number to ignore if you’re building agent eval today.

5. LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models

https://arxiv.org/abs/2603.28301 | GitHub

Vision-Language-Action models look impressive until you reword the instruction, and then performance drops 22 to 52 points across the seven configurations tested (0.6B to 7.5B parameters). LIBERO-Para isolates action phrasing from object references so you can see exactly where the model is reading words instead of meaning. The authors trace 80 to 96 percent of failures to the planning stage, not execution, and introduce PRIDE, a metric that quantifies paraphrase difficulty by semantic and syntactic distance.

🎥 4 Videos

1. World Models You Can Actually Interact With: Inside Moonlake

Interactivity changes what a world model has to learn, and that’s the thread Stanford’s Chris Manning and Fan-yun Sun pull on in this Latent Space conversation about Moonlake. The 67-minute episode gets into how the training signal differs from passive video models, and where this approach sits relative to Marble, Cosmos, and the gaming-data world models that have dominated the last quarter. Worth watching as a counterweight to the “world models = video generation” framing.

2. How OpenAI’s Codex Team Actually Builds with Codex

“For specs, we write like 10 bullets and that’s it,” and “our designers now write more code than eng did 6 months ago” are two of the throwaway lines in this 43-minute sit-down with Codex product lead Alex and developer experience lead Romain on Peter Yang’s channel. The conversation is unusually candid about what shipping without traditional specs and roadmaps looks like inside a team that lives on its own tools. Closest thing to a field report you’ll get on agent-native product development right now.

3. A Non-Engineer Runs Claude Code Across 15 Repos to Answer Every Customer Question

Al Chen is a field engineer at Galileo, an AI observability platform, and he has never held an engineering role. He has also built something most coding teams would be proud of: a Claude Code setup that queries 15 separate internal repositories, stitches in Confluence docs and customer-specific quirks, and delivers answers that previously required pulling an engineer off real work. The walkthrough covers his custom Claude Code commands, the sixteen-line sync script (written entirely by Claude Code) that pulls every repo’s main branch each morning, and the multi-source MCP pattern that lets a single question hit code, docs, and deployment notes in one pass. If you’ve been treating Claude Code as a coding assistant, this is the reframe that turns it into a customer support operating system.

4. From Chaos to Choreography: Multi-Agent Orchestration That Actually Works

Sandipan Bhaumik from Databricks opens with the line of the week: “Adding more agents isn’t adding more features. It’s building a distributed system.” His 26-minute AI Engineer talk covers the silent handoff failures, stale state, and untraceable decisions that show up once you scale from one agent to five, then walks through the orchestrator and choreography patterns Databricks uses in production. If you’re past the proof-of-concept stage, this is the talk you actually need.

📰 3 Curated Reads

1. Components of a Coding Agent

Ahead of AI

Components of A Coding Agent

In this article, I want to cover the overall design of coding agents and agent harnesses: what they are, how they work, and how the different pieces fit together in practice. Readers of my Build a Large Language Model (From Scratch) and Build a Large Reasoning Model (From Scratch…

Read more

3 months ago · 511 likes · 51 comments · Sebastian Raschka, PhD

Sebastian Raschka makes the case that what looks like a smarter model is usually a better harness around the same model. He breaks coding agents into six concrete pieces (live repo context, prompt shape and cache reuse, tool access, context reduction, structured session memory, and bounded subagents) and shows how they fit into a three-layer architecture of model, agent loop, and runtime support. Read this before you blame your model for behavior that’s actually a context engineering problem.

2. Your RAG Pipeline Is Overkill

Decoding AI Magazine

Your RAG Pipeline Is Overkill

We constantly fight a battle against the context window limit. You either compress your data until it loses meaning, or you build a massive infrastructure project just to read a few documents. Today, we look at a third option. We explore a pattern that allows models to read millions of tokens by treating data as an environment rather than an input…

Read more

3 months ago · 43 likes · 3 comments · Paul Iusztin

Paul Iusztin argues that recursive language models (RLMs) make most RAG pipelines unnecessary. The core idea is that the model never receives the giant document directly. Instead, the data lives outside the context as a REPL variable, and the model writes code to explore, filter, and recursively process it through `llm_query()` calls. Iusztin reports RLMs being tested up to 10 million tokens with GPT-5 and Qwen3-Coder, and lays out four scenarios (file parsing, codebase analysis, legal and financial work, research synthesis) where this approach beats stuffing chunks into a vector store.

3. Gemma 4 and What Makes an Open Model Succeed

Interconnects AI

Gemma 4 and what makes an open model succeed

Having written a lot of model release blog posts, there’s something much harder about reviewing open models when they drop relative to closed models, especially in 2026. In recent years, there were so few open models, so when Llama 3 was released most people were still doing research on Llama 2 and super happy to get an update. When…

Read more

3 months ago · 70 likes · Nathan Lambert

Benchmark numbers at release tell you almost nothing about which open models will actually get used, and Nathan Lambert uses the Gemma 4 launch to explain why. His five-factor framework (performance and size, country of origin, license, tooling at release, fine-tunability) maps onto the messy reality that ecosystem maturity often takes 18 months to catch up with a model launch. The note about Google moving to Apache 2.0 and the 30B dense model targeting the enterprise sweet spot is the part to underline.

🛠 2 Tools & Repos

1. mem0

https://github.com/mem0ai/mem0

Memory is the bottleneck for any agent that needs to remember anything across sessions, and mem0 is the fastest-moving open option for solving it. The framework manages user, session, and agent state through a single API, with self-hosted Python and TypeScript packages plus a managed cloud service if you want to skip the infra. The team reports 26% better accuracy than OpenAI Memory and roughly 90% lower token usage versus full-context approaches, which matches what people are seeing in production.

2. goose

https://github.com/aaif-goose/goose

Block built goose as a general-purpose AI agent in Rust, and it’s now part of the Agentic AI Foundation under the Linux Foundation. You get a desktop app, CLI, and API that work with 15+ LLM providers and over 70 MCP extensions, and you can use existing Claude, ChatGPT, or Gemini subscriptions instead of API keys. If you’ve been looking for a serious open alternative to the closed coding agent stack, this is the most active one right now.

🎓 1 Pick of the Week

How to Automate Your Life with Claude Code

Claude Code as a personal operating system, not as a coding tool. Hilary Gridley (former product leader) walks through how she runs her professional work and personal life through it as her primary interface. Her “anti-system system” leans on simple capture (an iPhone back-tap shortcut) and lets the model learn her preferences through observation rather than upfront configuration. The 10x impact framework for deciding what to automate is the part most people will steal, and the whole 51-minute walkthrough is reproducible on your own setup.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

TL;DR

What caught my attention this week:

• 📄 Papers: Streaming video generation hits 16 FPS, speculative sampling gets task-aware, and an autonomous medical AI scientist passes peer review

• 🎥 Videos: Sebastian Raschka maps 54+ LLM architectures, TurboQuant compresses KV cache to 3.5 bits, and two practical Claude Code tutorials

• 📰 Reads: Ethan Mollick on why AI interfaces matter more than models, Cameron Wolfe dissects LLM benchmarks, and Anthropic’s three primitives for context management

• 🛠 Tools: A biomimetic agent memory system with retain/recall/reflect, and an autonomous pentester that proves vulnerabilities with working exploits

• 🎓 Learning: Tsinghua’s multi-agent AI classroom turns any topic into an interactive lesson with AI teachers, students, and a shared whiteboard

📄 5 Papers

1. ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling

https://arxiv.org/abs/2603.25746 | GitHub

Real-time multi-shot video generation that maintains character consistency across scene transitions. ShotStream introduces a causal architecture with dual-cache memory (global context for inter-shot consistency, local context for intra-shot coherence) that enables ~16 FPS streaming, a 25x throughput improvement over bidirectional approaches. From the Kling AI Research team, this is the first system that makes interactive video storytelling feel responsive enough for real-time use.

2. Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models (HyDRA)

https://arxiv.org/abs/2603.25716 | GitHub

Video world models lose track of objects the moment they disappear behind something. HyDRA fixes this with a hybrid memory system that separates archival storage (for static scenes) from working memory (for active, occluded objects). The result: +5.5 PSNR improvement over commercial systems like WorldPlay on a new Dynamic Object Tracking benchmark. Also ships HM-World, the first large-scale video dataset dedicated to hybrid memory evaluation with exit-entry occlusion events.

3. TAPS: Task Aware Proposal Distributions for Speculative Sampling

https://arxiv.org/abs/2603.27027 | GitHub

Speculative decoding’s dirty secret: generic draft models waste tokens because they don’t match the downstream task distribution. TAPS trains task-specific draft models that align with the target model’s behavior on actual workloads, yielding ~26% acceptance length improvements over general-purpose drafters. Practical and immediately applicable if you’re running speculative decoding in production.

4. Towards a Medical AI Scientist

https://arxiv.org/abs/2603.28589

An autonomous research framework that generates clinical hypotheses, designs experiments, executes analyses, and writes papers. The system achieved 91% execution success rate versus GPT-5’s 60%, and one of its generated papers was accepted at ICAIS 2025 (36.8% acceptance rate). This is a concrete step toward AI that does science, not just assists with it.

5. PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

https://arxiv.org/abs/2603.25730 | GitHub

Training on long videos is expensive. PackForcing shows you don’t need to. By introducing hierarchical KV-cache management with a bounded 4GB memory budget, it achieves 24x temporal extrapolation: models trained on short clips generate coherent long videos. The spatiotemporal compression maintains temporal consistency while keeping inference memory constant regardless of video length.

🎥 4 Videos

1. 54+ LLM Architectures and 7 Attention Variants in One Visual Gallery

A visual map of how LLM architectures evolved from GPT-2 (2019) to Kimi K2.5 (2026), covering 54+ models from 270M to 1T parameters. The 38-minute deep dive compares 7 attention variants: standard MHA, Grouped-Query, Sliding-Window, Multi-Head Latent (DeepSeek’s MLA), Sparse, Gated, and Hybrid. Sebastian Raschka built the companion Architecture Gallery as an open resource, and this walkthrough is how you actually learn to read it.

2. TurboQuant: Compressing KV Cache to 3.5 Bits Per Channel

Random rotation plus scalar quantization can compress KV cache vectors to near-optimal distortion at 3.5 bits per channel. That’s the core of Google’s TurboQuant paper, broken down here by Karoly Zsolnai-Feher (Two Minute Papers). The practical result: cheaper LLM inference through aggressive cache compression without meaningful quality loss, with community review context and reproduction attempts included.

3. Building a Mobile Fitness App with Claude Code and Pencil in 16 Minutes

A three-phase workflow for building mobile apps without coding: define requirements conversationally, generate 8 UI screens with Pencil (pencil.dev), then build the working app with Claude Code. Peter Yang takes it from blank screen to a fitness tracker running on iOS via Expo Go in 16 minutes. Covers workout creation, live session tracking, calendar progress, and the path to App Store deployment.

4. The “Recording Mode” Trick for Privacy-Safe Claude Code Demos

A 2-minute clip with a clever idea: instead of maintaining separate anonymized demo environments, create a Claude Code skill called “recording on” that intercepts and anonymizes personal information in real-time. It tracks consistent mappings (person A stays person A), toggles on and off with zero friction, and works for B2B demos where you need to show live production data without exposing customer names or financial details.

📰 3 Curated Reads

1. Claude Dispatch and the Power of Interfaces

One Useful Thing

Claude Dispatch and the Power of Interfaces

AIs are already far more capable than most people realize. A large part of this so-called capability overhang comes not from the limits of AI (though, of course, they still have many limits), but from how people interact with it. The vast majority of people access AI through chatbots, and usually the free versions with less capable models. A chatbot is …

Read more

3 months ago · 551 likes · 26 comments · Ethan Mollick

Ethan Mollick (Wharton) argues the gap between AI capability and actual user experience is an interface problem, not a model problem. Current chatbot UIs impose cognitive costs that overwhelm productivity gains, particularly for less experienced workers. Three paths forward: specialized professional tools (the coding IDE model), meeting users on familiar platforms (WhatsApp, Slack), and dynamic interfaces where AI generates the right UI on the fly. Includes hands-on demos of Claude Dispatch and Cowork.

2. The Anatomy of an LLM Benchmark

Deep (Learning) Focus

The Anatomy of an LLM Benchmark

Throughout the history of AI research, progress has been measured—and accelerated—by high-quality benchmarks. AI is an empirical field that is driven by discovering interventions that improve performance on key benchmarks. For large language models (LLMs) in particular, creating useful benchmarks is hard due to rapidly advancing …

Read more

4 months ago · 73 likes · Cameron R. Wolfe, Ph.D.

LLM benchmarks break in predictable ways: MMLU has a 6.49% error rate (Virology hits 57%), and correcting those errors moved Llama-3.1-405B from 16th to 1st. Cameron Wolfe (Deep Learning Focus) maps the full lifecycle of how benchmarks are designed, saturate, and get replaced. The standout section covers Item Response Theory, which cuts evaluation costs 140-160x by selecting only the most informative test items.

3. Context Engineering for AI Agents: Memory, Compaction, and Tool Clearing

https://platform.claude.com/cookbook/tool-use-context-engineering-context-engineering-tools

Three composable primitives for managing context in long-running agents, each targeting a different type of bloat. Clearing drops re-fetchable tool outputs at zero inference cost (peak context: 173K tokens vs. 335K baseline). Compaction summarizes conversation history (169K peak, lossy). Memory persists knowledge across sessions by letting the agent write its own notes. Isabella He (Anthropic) includes a diagnostic framework: profile your agent’s token breakdown first (in the demo, 96.3% of tokens were stale file-read results), then pick the primitive that matches.

🛠 2 Tools & Repos

1. Hindsight

https://github.com/vectorize-io/hindsight

Agent memory that goes beyond conversation history. Hindsight provides three API primitives (retain, recall, reflect) that let agents learn, retrieve, and synthesize knowledge over time. Recall runs four parallel strategies (semantic, keyword, graph, temporal) with cross-encoder reranking. The reflect API generates new insights from existing memories, not just retrieval. SOTA on LongMemEval, 6.8K GitHub stars, MIT license, works with any LLM provider, and deploys via Docker or as an embedded Python library.

2. Shannon

https://github.com/KeygraphHQ/shannon

An autonomous AI pentester that reads your source code and executes real exploits against the running app. It builds a Code Property Graph to trace data flows from user input to dangerous sinks, then attacks those paths. Every reported vulnerability comes with a working proof-of-concept, not theoretical findings. Handles authentication complexity including 2FA/TOTP flows.

🎓 1 Pick of the Week

OpenMAIC: Tsinghua’s Multi-Agent AI Classroom

https://github.com/THU-MAIC/OpenMAIC

Type “teach me Python in 30 minutes” and OpenMAIC generates a full interactive classroom: slides with narration, quizzes with real-time grading, interactive HTML simulations, and AI agents playing teacher and student roles who lecture, discuss, and draw on a shared whiteboard. Built on LangGraph with a director agent that orchestrates turn-taking, it supports uploading PDFs for document-to-course conversion. 13.6K stars in three weeks, live demo at open.maic.chat, and the LangGraph-based multi-agent orchestration pattern is a reusable blueprint beyond education.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

TL;DR

What caught my attention this week:

• 📄 Papers: Diffusion-based OCR that’s 3x faster, a package manager for AI agent skills, world modeling from Monster Hunter, probability-aware RL clipping, and speculative shortcuts for agentic vision systems

• 🎥 Videos: Stripe’s internal AI coding agents at scale, building your own AI operating system from scratch, DeepSeek’s conditional memory for transformers, and going from Figma design to working code with Claude

• 📰 Reads: Sebastian Raschka’s visual taxonomy of attention mechanisms, Google’s training-free KV-cache compression via polar coordinates, and why on-device AI needs new architectures (not smaller cloud models)

• 🛠 Tools: ByteDance’s filesystem-based context database for AI agents, and Cornell’s free GPU architecture course

• 🎓 Learning: A beautifully maintained Claude Code cheat sheet that doubles as a feature discovery tool

📄 5 Papers

1. MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

https://arxiv.org/abs/2603.22458 | GitHub

What if OCR didn’t need to generate text one token at a time? This paper replaces autoregressive decoding with block-wise diffusion denoising, reframing document OCR as inverse rendering. The result: up to 3.2x faster throughput with a tunable speed-accuracy tradeoff depending on your pipeline needs. A proposed “Semantic Shuffle” benchmark shows the model genuinely reads visual structure rather than leaning on linguistic shortcuts.

2. SkillNet: Create, Evaluate, and Connect AI Skills

https://arxiv.org/abs/2603.04448 | GitHub

Think npm, but for AI agent skills. SkillNet treats agent capabilities as shareable, composable packages with a unified ontology for creating skills from heterogeneous sources (repos, docs, logs, prompts) and connecting them through dependency graphs. The framework delivers a 40% improvement in average rewards and 30% fewer execution steps across ALFWorld, WebShop, and ScienceWorld. With 200,000+ skills in the repository and a Python toolkit plus Open Access API, this could become critical infrastructure for the tool-using agent ecosystem.

3. WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG

https://arxiv.org/abs/2603.23497 | GitHub

Most world-modeling datasets treat video prediction as frame interpolation. WildWorld takes a different approach: 108 million frames automatically collected from Monster Hunter: Wilds, with per-frame action labels (450+ semantically meaningful actions), character skeletons, world states, camera poses, and depth maps. The key insight is decomposing dynamics into action, state, and pixels separately, giving world models the causal structure they need to simulate rather than interpolate.

4. BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning

https://arxiv.org/abs/2603.04918 | GitHub

PPO’s fixed clipping bounds have a quiet failure mode: they disproportionately suppress high-advantage, low-probability actions, which are exactly the exploratory tail strategies you want to preserve in reasoning tasks. BandPO replaces static bounds with dynamic, probability-aware intervals derived from f-divergence constraints, formulated as a convex optimization problem with closed-form solutions. Tested on Qwen2.5 and Llama3 for mathematical reasoning, it consistently outperforms canonical GRPO clipping while preventing entropy collapse. A drop-in replacement for anyone doing GRPO-style training post-DeepSeek-R1.

5. SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning

https://arxiv.org/abs/2603.23483 | GitHub

Agentic vision systems are slow because every query triggers a full sequential tool-use chain, even when the answer is straightforward. SpecEyes uses a lightweight gating mechanism (based on top-K logit gaps) to predict when the expensive pipeline can be short-circuited. The result: up to 3.35x speedup (averaging 1.4-1.7x across benchmarks) with accuracy maintained or improved by up to 6.7%. The gating requires no labeled routing data, making it deployable without per-task annotation on any production agentic vision system where latency matters.

🎥 4 Videos

1. Building a Full-Stack AI Operating System from Scratch

A three-layer architecture for a personal AI platform: webhook-driven triggers, scheduled workflows (cron-style recurring tasks), and an autonomous agent layer with persistent context and reusable skills. Dave Ebbelaar builds the entire system on FastAPI, Celery, Redis, and Docker. This 28-minute walkthrough gives you the blueprint instead of cloning random agent repos.

2. DeepSeek’s Engram: Adding Conditional Memory to Transformers

DeepSeek’s Engram module adds O(1) lookup memory to transformers by modernizing n-gram embeddings as a new sparsity dimension alongside Mixture-of-Experts. Dr. Karoly Zsolnai-Feher (Two Minute Papers) explains how it outperforms comparable MoE baselines at 27B parameters on both parameter and compute budgets. The surprise: the biggest gains show up not in knowledge retrieval (MMLU +3.4) but in reasoning (BBH +5.0, ARC-Challenge +3.7), because offloading pattern reconstruction to memory frees attention depth for harder tasks.

3. From Figma Design to Working Code with Claude Code and MCP

Figma mockup to working website in 15 minutes, FigJam flowchart to a working game, then exporting code back to Figma as editable components. Felix Lee (designer at ADPList) demonstrates the full design-to-code loop using Claude Code with Figma MCP in this 50-minute session hosted by Peter Yang. The core insight: MCP reads every color, spacing value, and component variant directly from Figma, eliminating the translation loss in design-to-dev handoffs.

4. How Stripe Ships 1,300 AI-Written Pull Requests Per Week

Stripe built internal AI coding agents called “minions” that now ship roughly 1,300 PRs per week. Steve Kaliski (software engineer at Stripe) walks through the architecture: Goose (Block’s open-source agent harness) with cloud dev environments, activated from Slack via emoji reactions. The key takeaway: Stripe’s existing investment in developer tooling (CI, testing, linting) made agent adoption frictionless, because good DX for humans turns out to be good DX for agents too. Non-engineers at Stripe now use minions to ship code.

📰 3 Curated Reads

1. A Visual Guide to Attention Variants in Modern LLMs

Ahead of AI

A Visual Guide to Attention Variants in Modern LLMs

I had originally planned to write about DeepSeek V4. Since it still hasn’t been released, I used the time to work on something that had been on my list for a while, namely, collecting, organizing, and refining the different LLM architectures I have covered over the past few years…

Read more

4 months ago · 267 likes · 6 comments · Sebastian Raschka, PhD

Seven attention mechanism families in one visual taxonomy, from classic Multi-Head Attention through GQA, DeepSeek’s Multi-Head Latent Attention, Sliding Window Attention, and hybrid architectures mixing transformers with linear or state-space modules. Sebastian Raschka diagrams how queries, keys, and values interact in each variant, how KV-cache mechanics change, and how memory growth curves differ. The practical payoff: a clear framework for choosing the right attention mechanism based on your model scale and deployment constraints, with Gemma 3 and DeepSeek V2 as case studies.

2. TurboQuant: Redefining AI Efficiency with Extreme Compression

https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/

Google Research introduces a training-free, data-agnostic compression algorithm that achieves 3-bit KV-cache quantization without accuracy loss. The trick: PolarQuant converts data vectors from Cartesian to polar coordinates (concentrating angle patterns predictably), then QJL reduces residual errors to single sign bits via random projections. On H100 GPUs, TurboQuant delivers a 6x reduction in KV memory and up to 8x performance gains over 32-bit unquantized baselines. Unlike most quantization work, this targets the KV-cache specifically (the bottleneck that grows with context length) and requires zero calibration data or fine-tuning.

3. The Future of On-Device AI

Artificial Intelligence Made Simple

The Future of On-Device AI

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction…

Read more

4 months ago · 55 likes · 1 comment · Devansh

Devansh argues the real bottleneck for on-device AI is memory bandwidth, not compute. Using Liquid AI’s LFM2 (1.2B parameters, runs on Samsung Galaxy S25) as a case study, the piece shows why shrinking data-center models is the wrong approach. LFM2 uses 10 gated short convolutions plus 6 grouped-query attention blocks, cutting peak cache to 192 MB at 32K tokens (versus Llama 3.2 1B’s 524 MB). It matches Qwen3-1.7B on benchmarks despite having 42% fewer parameters and runs at 70 tokens per second on a phone CPU. The thesis: the field needs device-native architectures designed from scratch, not miniaturized cloud models.

🛠 2 Tools & Repos

1. OpenViking: A Context Database for AI Agents

https://github.com/volcengine/OpenViking

ByteDance’s open-source solution for the “stuff everything into the prompt” problem. OpenViking organizes agent context (memories, resources, skills) into a navigable filesystem hierarchy with three-tier demand-based loading, so agents only consume tokens for what they actually need. It combines directory-based navigation with semantic search, auto-compresses conversations into long-term memory, and provides visualization of retrieval trajectories for debugging. 19.1K stars, Apache 2.0, supports major LLM providers via LiteLLM.

2. Cornell Virtual Workshop: GPU Architecture Fundamentals

https://cvw.cac.cornell.edu/gpu-architecture/gpu-characteristics/design

Free, NSF-funded course from Cornell’s Center for Advanced Computing covering why GPUs work the way they do: transistor allocation tradeoffs, memory hierarchies, parallelization design choices, and the practical implications for your code. Modules covering fundamentals through V100 and RTX 5000 deep dives, with exercises. No parallel programming experience assumed. If you call `.cuda()` daily but lack a mental model for what happens underneath, start here.

🎓 1 Pick of the Week

Claude Code Cheat Sheet

https://cc.storyfox.cz/

This single page Claude code cheat sheet covers 100+ commands across 8 color-coded sections: keyboard shortcuts, ~40 slash commands, MCP server configuration, memory and files, workflows, config, skills and agents, and CLI flags. Commands like `/btw` for side questions without derailing context, `/schedule` for cloud-scheduled tasks, and git worktree isolation with sparse checkout are buried in docs but surfaced here at a glance. Auto-detects Mac versus Windows, prints cleanly to A4, works offline, and updates daily as Claude Code evolves.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

Subscribe now

TL;DR

What caught my attention this week:

• 📄 Papers: Low-bit attention training, efficient VLMs, scientific discovery shortcuts, robot planning in 8 tokens, and reasoning models that can’t control their own thoughts

• 🎥 Videos: Sakana AI’s evolved transformers, Turbopuffer’s post-RAG retrieval architecture, Figma’s Claude Code design pipeline, and DeepMind reflects on AlphaGo’s decade of impact

• 📰 Reads: Nathan Lambert on why the open model gap will widen (not close), production RAG done right, and diagnosing failures across 192-GPU training clusters

• 🛠 Tools: ByteDance’s open-source SuperAgent platform and OpenAI’s AI-powered LaTeX editor

• 🎓 Learning: A community-built field manual for getting the most out of Claude Code

📄 5 Papers

1. SageBwd: A Trainable Low-bit Attention

https://arxiv.org/abs/2603.02170 | GitHub

SageAttention already had 3,200+ GitHub stars for speeding up inference. Now SageBwd extends quantized attention to training by quantizing 6 of 7 attention matrix multiplications in the backward pass. The result: up to 1.67x speedup over FlashAttention2 with negligible loss difference (2.561 vs 2.563 at 260K tokens per step). If you’re spending money on attention compute during training, this is the paper to read.

2. Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoder

https://arxiv.org/abs/2603.06569

What if you replaced CLIP/SigLIP vision encoders with one initialized from a plain text LLM? Tencent AI Lab tried it. Their 8B model outperforms Qwen3-VL-8B and InternVL3.5-8B on document understanding, visual knowledge, and video reasoning, hitting 96.2 on DocVQA and 90.5 on ChartQA. Vision encoder architecture matters more than you’d think. Sometimes the answer is just using a language model.

3. MOOSE-Star: Unlocking Tractable Training for Scientific Discovery

https://arxiv.org/abs/2603.03756

Scientific hypothesis discovery with LLMs has an exponential search problem. MOOSE-Star (from MiroMind AI) reduces combinatorial O(N^k) search to roughly logarithmic via hierarchical decomposition, hitting 100% success rate at around 6,000 inference calls where brute-force saturates at 41.3%. Also releases TOMATO-Star, a dataset of 108,717 decomposed papers for benchmarking. The complexity reduction alone makes previously intractable hypothesis spaces searchable.

4. CompACT: A Compact Discrete Tokenizer for Latent World Model Planning

https://arxiv.org/abs/2603.05438

Robot planning typically requires hundreds of visual tokens per observation. CompACT (from POSTECH and KAIST, accepted at CVPR 2026) compresses that down to as few as 8 discrete tokens, making world-model planning 40x faster. Navigation planning drops from 178 seconds to under 6 seconds with competitive accuracy. This is what makes real-time robotic planning actually feasible.

5. Reasoning Models Struggle to Control their Chains of Thought

https://arxiv.org/abs/2603.05706

Can reasoning models actually control their chain of thought? Researchers from NYU, OpenAI, UCL, and UPenn tested 13 models. Claude Sonnet 4.5 achieves only 2.7% CoT controllability (versus 61.9% output controllability). DeepSeek R1 scores 0.1%. The safety implication: if models can’t steer their reasoning strategically, they also can’t easily hide deceptive reasoning from monitors. Matters more for what it implies than what it measures.

🎥 4 Videos

1. Sakana AI’s Open-Ended Evolution of Transformers with Robert Lange

LLMs combined with evolutionary algorithms for open-ended program search. That’s Shinka Evolve from Sakana AI, discussed by Robert Lange on Machine Learning Street Talk. Why “solving the wrong problem” sometimes leads to better architectures. How evolutionary pressure discovers novel transformer variants. What open-endedness means for AI research beyond benchmarks. Covers the gap between optimizing a known objective and discovering objectives worth optimizing.

2. Retrieval After RAG: Hybrid Search, Agents, and Database Design with Turbopuffer’s Simon Eskildsen

What comes after the first wave of RAG implementations? Hybrid search architectures, why vector-only retrieval hits a ceiling, agent-driven retrieval reshaping database design. Simon Eskildsen (Turbopuffer founder) walks through real case studies from Cursor and Notion on Latent Space. If you’ve built a RAG system and hit a quality wall, this is where you go next.

3. How Figma Engineers Sync Designs with Claude Code and Codex

MCP-based tooling that creates a continuous sync between design and code. Figma’s Gui Seiz and Alex Kern show their team’s actual production workflow using Claude Code and Codex. Not a concept demo. The design handoff becomes a two-way pipeline. Worth 40 minutes if your team still does screenshot-to-implementation.

4. 10 Years of AlphaGo: The Turning Point for AI

AlphaGo’s techniques propagated into protein structure prediction, materials science, and chip design. Google DeepMind’s Thore Graepel and Pushmeet Kohli trace the full decade of impact beyond the Go match itself. Covers how one system’s ideas became foundational building blocks across scientific domains.

📰 3 Curated Reads

1. What comes next with open models

Interconnects AI

What comes next with open models

2025 was the year where a lot of companies started to take open models seriously as a path to influence in the extremely valuable AI ecosystem — the adoption of a strategy that was massively accelerated downstream of DeepSeek R1’s breakout success. Most of this is being done as a mission of hope, principle, or generosity…

Read more

4 months ago · 59 likes · 12 comments · Nathan Lambert

The open-closed model gap will widen, not close. Nathan Lambert’s reframing: the real opportunity isn’t chasing frontier capability but building small, specialized models that are 10x faster and 100x cheaper. Introduces the “open models as sub-agents” framing where open-weight models handle specialized tasks within larger systems. Changes how you evaluate open models if you’ve been benchmarking them against GPT-5.

2. Production RAG: Learning from Scratch Done Right

Decoding AI Magazine

Why Most RAG Tutorials Fail You

Paul: Today, the stage belongs to Priya, a Senior Software Architect who’s spent years shipping production-scale systems at Publicis Sapient and Tesco…

Read more

4 months ago · 46 likes · 5 comments · Priya

Most RAG tutorials optimize for demos, not production. This piece on Paul Iusztin’s Decoding AI (by guest contributor Priya) walks through a 4-phase production RAG system: ingestion, retrieval, generation, serving. Uses Postgres and pgvector with explicit control flow and data lineage. The core insight: a bad chunk doesn’t throw an exception, it just hallucinates an answer three steps later. If your RAG prototype works in notebooks but fails in production, start here.

3. How to Diagnose Failures in Large AI Training Clusters

Artificial Intelligence Made Simple

How to Diagnose Failures in Large AI Training Clusters

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction…

Read more

4 months ago · 35 likes · Devansh

AI agents autonomously executing diagnostic runbooks against a unified Prometheus TSDB. Devansh details five case studies across multi-GPU clusters with quantified results: 30% throughput recovery, checkpoint restore penalties reduced from 1.0% to 0.15%. Not theoretical. Includes the actual diagnostic architecture and failure patterns you’d hit at this scale.

🛠 2 Tools & Repos

1. deer-flow (ByteDance)

https://github.com/bytedance/deer-flow

ByteDance’s open-source SuperAgent platform (MIT license, 31K+ stars) got a ground-up v2.0 rewrite that hit #1 on GitHub Trending. Ships as a complete deployable platform, not a framework you wire together. Web UI, Docker-sandboxed execution, persistent cross-session memory, parallel sub-agent spawning, messaging integrations (Telegram, Slack, Feishu). Built on LangGraph. For teams that want a working multi-agent system without assembling one from parts.

2. Prism (OpenAI)

https://prism.openai.com/

OpenAI’s browser-based LaTeX editor with GPT-5.2 integrated inline. Free tier: unlimited projects, compiles, and collaborators (Pro at $7/mo for unlimited AI features). Highlight text, ask the AI to rewrite or formalize, and it compiles in real time. Zotero integration, image-to-LaTeX, voice mode for dictating equations. Best for refining existing papers. Won’t generate structure from a blank page. No Git integration yet, which is the main gap versus Overleaf.

🎓 1 Pick of the Week

claude-code-best-practice

https://github.com/shanraisshan/claude-code-best-practice

How should you actually use Claude Code day-to-day? Official docs don’t answer that. This community-built field manual does (17,600+ stars, actively maintained). 40+ actionable tips across 8 categories, comparative reports against other tools, community workflow implementations. Includes a working `.claude/` directory you can clone. The “billion-dollar questions” section names what the community still hasn’t figured out. If you’re already using Claude Code and want to move from casual to systematic, bookmark this.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here? The Tokenizer is my resource-focused newsletter edition where I curate the best AI/ML papers, videos, articles, tools, and learning resources so you don’t have to sift through the noise. Subscribe to Gradient Ascent for the full experience.

TL;DR

What caught my attention this week:

• 📄 Papers: Smarter reasoning graphs, temporal backdoors in tool-using LLMs, spatial reasoning benchmarks, privacy advantages of diffusion language models, and versatile video editing

• 🎥 Videos: Hardware constraints shaping LLM architecture, brand-consistent image generation with Midjourney, a complexity taxonomy for AI agents, and a critical look at vibe coding

• 📰 Reads: Statistical rigor for LLM evaluations, the infrastructure costs of long-context inference, and the current state of open-weight models catching up to closed frontiers

• 🛠 Tools: A 101k-star collection of LLM applications and a versioned API documentation system for coding agents

• 🎓 Learning: Karpathy’s tool that lets AI agents run autonomous ML experiments overnight

📄 5 Papers

1. RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoning

https://arxiv.org/abs/2603.05818

Graph of Thoughts is powerful but expensive, treating every reasoning node equally regardless of difficulty. RouteGoT fixes this by adaptively routing compute across graph nodes, skipping the heavy lifting where it isn’t needed. The result: 8.1 percentage points more accurate than AGoT while using 79.1% fewer output tokens.

2. Sleeper Cell: Injecting Latent Malice Temporal Backdoors into Tool-Using LLMs

https://arxiv.org/abs/2603.03371

Here’s a security scenario worth losing sleep over: backdoors implanted in tool-using LLMs that only activate under specific temporal conditions. The model maintains state-of-the-art performance on benign tasks and evades standard safety evaluations, until a particular time trigger flips the switch. A sobering look at the gap between current evaluation practices and actual deployment safety.

3. SpatialText: A Pure-Text Cognitive Benchmark for Spatial Understanding in LLMs

https://arxiv.org/abs/2603.03002

Do LLMs actually understand spatial relationships, or are they just pattern-matching on word co-occurrences? SpatialText isolates spatial reasoning from visual shortcuts using pure text, and the results aren’t flattering. Current models lean heavily on linguistic heuristics rather than building coherent spatial representations, which matters for anyone building systems that need to reason about physical space.

4. Characterizing Memorization in Diffusion Language Models

https://arxiv.org/abs/2603.02333

Diffusion-based language models turn out to have a meaningful privacy advantage over autoregressive ones. This paper shows they exhibit substantially lower memorization-based leakage of personally identifiable information. If you’re building generative systems where training data sensitivity matters (medical, legal, financial), this distinction between generation architectures is worth understanding.

5. Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance

https://arxiv.org/abs/2603.02175 | GitHub

Video editing that combines instruction-based and reference-based approaches in one pipeline. Kiwi-Edit constructs high-fidelity training data using synthetic reference scaffolds, sidestepping the usual bottleneck of paired video editing datasets. The result is a system that handles both “make the sky purple” style instructions and “make it look like this reference” editing in a single model.

🎥 4 Videos

1. How Hardware Constraints Are Shaping Modern LLM Architecture

Julia Turc explores how physical hardware realities (memory bandwidth, compute density, interconnect speeds) are actively driving architectural decisions in modern LLMs. If you’ve wondered why certain design choices keep showing up across different labs, the answer often starts with what the silicon can actually do efficiently.

2. Mastering Midjourney: Consistent Brand Imagery Without Complex Prompts

On How I AI, AI creative director Jamey Gannon walks through her workflow for generating images that maintain consistent brand identity across multiple Midjourney outputs using style references, personalization codes, and mood boards. Useful for anyone who’s gotten great individual images from AI tools but struggled to make a cohesive visual set for a brand, product, or campaign.

3. The 5 Levels of AI Agent Complexity

Dave Ebbelaar breaks down AI agents into five distinct complexity levels, from simple single-tool agents to sophisticated multi-agent orchestration systems. Helpful framing for teams trying to scope what kind of agent they actually need (often simpler than they think) and understanding the engineering effort each level demands.

4. Vibe Coding is a Slot Machine

Machine Learning Street Talk sits down with Jeremy Howard (fast.ai) to examine whether AI coding assistants are genuinely improving developer productivity or creating a false sense of progress. Howard’s central argument: if you outsource all your thinking to computers, you stop building the competence that makes you effective. The kind of critical examination teams need before making investment decisions around AI-assisted development tooling.

📰 3 Curated Reads

1. Applying Statistics to LLM Evaluations

Deep (Learning) Focus

Applying Statistics to LLM Evaluations

Research on large language models (LLMs) is empirically driven. For this reason, model evaluations play a pivotal role in the field’s progress. We improve models by making changes, evaluating them, and iterating. Despite their foundational role, however, evaluations are usually handled in a naive manner. In most cases, we just test a mod…

Read more

4 months ago · 66 likes · 2 comments · Cameron R. Wolfe, Ph.D.

Cameron R. Wolfe from Deep (Learning) Focus tackles a problem most benchmarking papers quietly ignore: the statistical validity of LLM evaluation results. Key finding worth internalizing: clustered standard errors can increase reported uncertainty by 3x, and the Central Limit Theorem becomes unreliable with small sample sizes. If you’ve ever looked at a leaderboard and wondered “is this difference actually meaningful?”, this article gives you the tools to answer that.

2. How Long Context Inference Is Rewriting the Future of Transformers

Artificial Intelligence Made Simple

How Long Context Inference Is Rewriting the Future of Transformers

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction…

Read more

4 months ago · 56 likes · 2 comments · Devansh

AI Made Simple quantifies what long-context windows actually cost in production. A 70B-parameter model serving 59 concurrent users at 4K context drops to just 1 user at 128K context. The article covers the engineering responses, including Multi-Head Latent Attention (MLA) achieving 93.3% cache reduction. Practical reading for anyone deploying models where context length isn’t just a spec sheet number but a capacity planning constraint.

3. A Dream of Spring for Open-Weight LLMs

Ahead of AI

A Dream of Spring for Open-Weight LLMs: 10 Architectures from Jan-Feb 2026

If you have struggled a bit to keep up with open-weight model releases this month, this article should catch you up on the main themes…

Read more

5 months ago · 183 likes · 9 comments · Sebastian Raschka, PhD

Sebastian Raschka surveys the current open-weight landscape across 10 models and finds the gap to closed frontiers narrowing fast. GLM-5 now benchmarks on par with GPT-5.2, Gemini Pro 3, and Claude 4.6 Opus. On the inference side, Step 3.5 Flash hits 100 tokens/sec compared to DeepSeek’s 33. A well-structured overview for tracking which open models are actually competitive and where the remaining gaps lie.

🛠 2 Tools & Repos

1. Awesome LLM Apps

https://github.com/Shubhamsaboo/awesome-llm-apps

At 101k stars, this collection of LLM application examples covers RAG implementations, AI agents (single and multi-agent teams), MCP integration patterns, voice AI, and fine-tuning guides. More useful as a reference architecture library than a tutorial. When you’re building something new, check here first to see how others have solved similar problems.

2. Context Hub

https://github.com/andrewyng/context-hub

From Andrew Ng’s team, Context Hub gives coding agents access to curated, versioned API documentation instead of letting them hallucinate library APIs. Features search and fetch, language-specific variants, persistent annotations, and feedback loops so the documentation improves over time. At 3.5k stars, it’s gaining traction with teams building custom coding agents that need reliable API knowledge.

🎓 1 Pick of the Week

Autoresearch

https://github.com/karpathy/autoresearch

Andrej Karpathy’s latest side project is a Python tool built around three core files (prepare.py, train.py, program.md) that lets an AI agent run autonomous ML experiments on a single GPU. The agent proposes hypotheses, writes training code, runs experiments with a 5-minute budget each, and iterates based on results. Leave it running overnight and wake up to a stack of completed experiments. At 23.6k stars within days of release, it’s clearly struck a nerve. Sparks of recursive self-improvement, indeed.

Thanks for reading The Tokenizer! If you found this valuable, please share it with your colleagues and consider subscribing to Gradient Ascent for more AI insights.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best papers, videos, articles, tools, and learning resources from across the AI landscape. Consider it your weekly dose of everything you need to stay ahead in machine learning.

I’m teaching ML & Generative AI System Design on Feb 28th / March 1st with Packt.

We’ll cover AI systems that use RAG and traditional ML design principles for building solid AI products: making systems reliable, measuring what matters, and designing architectures that work in production.

Through live discussions, guided exercises, and design sprints, you’ll practice solving system-level AI problems and walk away with frameworks you can apply immediately at work.

Use code FLASH40 for 40% off:

Learn AI System Design

What topics/problems would you most want covered in a system design workshop? Drop a comment or DM me.

TL;DR

What caught my attention this week:

• 📄 Papers: Task handoff is not the same as authority transfer, video models tested on physics not aesthetics, memory that evolves without overfitting, gated reasoning that knows when to stop, and feature-level coding benchmarks exposing real agent limitations

• 🎥 Videos: Jeff Dean on AI’s Pareto frontier, comprehensive 2026 state of AI breakdown, building custom dev tools instead of buying SaaS, and practical context engineering for agents

• 📰 Reads: Rubric-based rewards for subjective domains, recursive language models that call themselves like functions, and why taste matters for generative AI

• 🛠 Tools: Reasoning implementations from scratch, comprehensive LLM learning roadmap

• 🎓 Learning: Karpathy’s microGPT strips GPT to its mathematical essence in pure Python

📄 5 Papers

FeatureBench: Benchmarking Agentic Coding for Complex Feature Development

https://arxiv.org/abs/2602.10975 | GitHub

Claude 4.5 Opus achieves 74.4% on SWE-bench but drops to 11.0% on FeatureBench. The difference? SWE-bench tests bug fixes within single pull requests, while FeatureBench evaluates end-to-end feature development spanning multiple commits and PRs across development timelines. Using a test-driven method that traces from unit tests along dependency graphs, the benchmark automatically derives 200 feature-level tasks from 24 repositories while ensuring other features remain functional after separation. This exposes the gap between fixing localized issues and actually building new capabilities.

Intelligent AI Delegation

https://arxiv.org/abs/2602.11865

Existing delegation methods run on heuristics that collapse when environments change or sub-agents fail. Intelligent AI Delegation reframes the problem: passing a task is not the same as transferring authority, responsibility, and accountability, and conflating the two is where multi-agent systems break down. The framework draws from principal-agent theory in economics, assumes zero trust at every delegation boundary, and applies to both human and AI delegators across complex agent networks. Misalignment, reward gaming, and deceptive behavior all compound as agents delegate to other agents. The paper is trying to define the protocol layer the agentic web needs before it can safely scale.

RISE-Video: Can Video Generators Decode Implicit World Rules?

https://arxiv.org/abs/2602.05986 | GitHub

Video models produce visually impressive outputs, but can they reason about physics? RISE-Video shifts evaluation from aesthetics to cognitive understanding with 467 human-annotated samples spanning eight categories. The benchmark tests whether models grasp implicit constraints like spatial dynamics, temporal consistency, physical rationality, and causality. Testing 11 state-of-the-art models revealed pervasive failures when simulating complex scenarios under implicit rules, exposing the gap between visual fidelity and genuine world understanding.

UMEM: Unified Memory Extraction and Management Framework for Generalizable Memory

https://arxiv.org/abs/2602.10652 | GitHub

Memory-enabled agents typically optimize management while treating extraction as static, accumulating instance-specific noise rather than robust insights. UMEM jointly optimizes both through Semantic Neighborhood Modeling, evaluating memory utility across clusters of semantically related queries rather than individual instances. Trained with neighborhood-level marginal utility rewards via GRPO, the approach achieves up to 10.67% improvement on multi-turn tasks while maintaining monotonic growth during continuous evolution. Agents that actually learn from experience rather than just retrieve logs.

When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning

https://arxiv.org/abs/2602.10560

Long-context reasoning faces critical issues: memory explodes from indiscriminate updates on evidence-free chunks, and loops continue unnecessarily after gathering sufficient evidence. GRU-Mem introduces update and exit gates controlled by text. Memory updates only when the update gate opens, and the loop terminates immediately when the exit gate opens. Trained with end-to-end RL using separate rewards for correct updating and exiting behaviors, GRU-Mem outperforms vanilla MemAgent with up to 4x inference speed acceleration.

🎥 4 Videos

Owning the AI Pareto Frontier — Jeff Dean

Jeff Dean unpacks the Pareto frontier in AI scaling, balancing compute, energy efficiency and model performance. Google’s Chief AI Scientist discusses the unification of their AI teams and why distillation is becoming the engine behind efficient models. Essential viewing for understanding the tradeoffs between model capability and practical deployment constraints.

State of AI in 2026: LLMs, Coding, Scaling Laws & Agents

Nathan Lambert and Sebastian Raschka join Lex Fridman to break down where we actually stand. They cover the reality of coding agents (echoing FeatureBench findings), scaling laws beyond just “more compute,” and practical challenges of building reasoning models. Two of the best technical communicators in AI deliver a comprehensive status report on 2026’s landscape.

DIY Dev Tools: The Shift to “Build” Over “Buy”

CJ Hess demonstrates “Flowy,” a custom tool he built to visualize coding plans. Instead of relying on static Markdown or finicky diagrams, he created a system where Claude generates JSON that renders into interactive flowcharts and UI mockups. When AI can write the code, building bespoke internal tools optimized for your specific workflow is often faster and better than buying generic SaaS.

Effective Context Engineering for AI Agents

Dave Ebbelaar explains why agents fail not from bad instructions but from poor context management. In short, context engineering is king. He offers practical strategies for structuring workflows and managing the context window as a limited resource rather than a magic bucket.

📰 3 Curated Reads

Rubric-Based Rewards for RL

Deep (Learning) Focus

Rubric-Based Rewards for RL

Many of the recent capability gains in large language models (LLMs) have been a product of advancements in reinforcement learning (RL). In particular, RL with verifiable rewards (RLVR) has drastically improved LLM capabilities by using rules-based, deterministic correctness checks (e.g., passing the test cases for a coding problem…

Read more

5 months ago · 64 likes · 5 comments

Cameron Wolfe explains that while RL has excelled in domains with clear answers like math and code, Rubric-Based RL uses LLMs as judges with strict rubrics to provide reward signals in subjective domains. This bridges the gap that could allow training reasoning models for writing, creativity, and strategy rather than just problems with verifiable solutions.

Recursive Language Models: To the Rescue

WHERE MACHINES THINK

Recursive Language Models: Can a simple method potentially unlock new regime of inference-time scaling?

IN THE FALL OF 2024, there were concerns that neural scaling laws were saturating. Throwing more compute and data to train ever-bigger large language models (LLMs) was showing diminishing returns. And then, OpenAI released its o1 series of “reasoning” models, unlocking an entirely new way to improve the performance of LLMs. Ope…

Read more

5 months ago · 17 likes · 1 comment · Anil Ananthaswamy

What if an LLM could call itself like a function? Recursive Language Models decompose complex prompts and recursively generate their own context. This architectural shift treats models less like text predictors and more like computer programs with call stacks, handling complexity through recursion rather than scale.

Taste for Makers

https://paulgraham.com/taste.html

Paul Graham breaks down the universal principles of good design across disciplines like math, coding, and architecture. For anyone building AI products today, treating “quality” as an objective, engineerable standard rather than a vague feeling is a massive competitive advantage.

🛠 2 Tools & Repos

Reasoning from Scratch (Chapter 7 Code)

https://github.com/rasbt/reasoning-from-scratch/blob/main/ch07/01_main-chapter-code/ch07_main.ipynb

Sebastian Raschka’s notebook implements Reinforcement Learning with Group Relative Policy Optimization (GRPO) from scratch. If you want to understand how models like DeepSeek-R1 work under the hood, this walks through the actual mechanics rather than hiding behind abstractions. Essential for anyone implementing reasoning capabilities.

LLM Course Roadmap

https://github.com/mlabonne/llm-course

Maxime Labonne’s repository remains the gold standard for self-learning. This curated roadmap takes you from fundamentals to fine-tuning your own models, recently updated with sections on agentic workflows and evaluation. The curriculum balances theoretical explanations with practical notebooks and hands-on projects mirroring real-world applications.

🎓 1 Pick of the Week

microGPT

https://karpathy.github.io/2026/02/12/microgpt/

Andrej Karpathy stripped GPT to its absolute core: 243 lines of pure Python with zero dependencies. No PyTorch, no NumPy, no frameworks. Just the full algorithmic content needed for training and inference: dataset handling, tokenizer, autograd engine, GPT-2-like architecture, Adam optimizer, training loop, and inference loop. The code fits perfectly across three columns and represents a decade-long obsession to simplify LLMs to bare essentials. Run it from a single file, understand every mathematical operation, see exactly how attention works without abstraction layers hiding the mechanics. For anyone who wants to truly understand transformers rather than just use them, this is the definitive resource.

Thanks for reading The Tokenizer! If you found something useful here, share it with someone who might benefit. And if you want more curated insights like this, consider subscribing to Gradient Ascent.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best papers, videos, articles, tools, and learning resources from across the AI landscape. Consider it your weekly dose of everything you need to stay ahead in machine learning.

I’m teaching ML & Generative AI System Design on Feb 28th / March 1st with Packt.

We’ll cover AI systems that use RAG and traditional ML design principles for building solid AI products: making systems reliable, measuring what matters, and designing architectures that work in production.

Through live discussions, guided exercises, and team-based design sprints, you’ll practice solving system-level AI problems and walk away with frameworks you can apply immediately at work.

Use code FLASH40 for 40% off:

Register for the Workshop

Questions? Drop a comment or DM me.

TL;DR

What caught my attention this week:

📄 Papers: Code-as-image achieving 8x compression, speculative decoding beating EAGLE-3 by 2.5x, video models learning from YouTube demonstrations, and self-optimizing RL systems

🎥 Videos: Systematic Claude Code workflows from Anthropic users, Stanford on agents and RAG patterns, visual MoE explanations, and vector indexing methods

📰 Reads: Why creativity can’t be interpolated, 10x cheaper tokens through prompt caching, and 12x faster MoE training with Unsloth

🛠 Tools: Language extraction for multilingual text and comprehensive LLM evaluation frameworks

🎓 Learning: Google’s viral Paper Banana for interactive research paper exploration

📄 5 Papers

Demo-ICL: In-Context Learning for Procedural Video Knowledge Acquisition

https://arxiv.org/abs/2602.08439 | GitHub

Most video benchmarks test static knowledge rather than whether models can actually learn from demonstrations. Demo-ICL tackles this directly with a benchmark built from 1200 instructional YouTube videos. The system provides models with text or video demonstrations, then asks them to answer questions about target videos by applying what they learned from the examples. The two-stage training approach combines video-supervised fine-tuning with information-assisted preference optimization, improving how models extract and apply procedural knowledge from demonstrations.

DFlash: Block Diffusion for Flash Speculative Decoding

https://arxiv.org/abs/2602.06036 | GitHub

Autoregressive models decode sequentially, creating a latency bottleneck that speculative decoding helps address. DFlash uses a lightweight block diffusion model that generates draft tokens in parallel rather than one at a time. By conditioning the draft model on context features from the target LLM, it maintains high acceptance rates while generating entire blocks simultaneously. The system achieves over 6x lossless acceleration across various tasks, delivering up to 2.5x higher speedup than EAGLE-3.

CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding

https://arxiv.org/abs/2602.01785 | GitHub

Text-based code processing creates a linear scaling problem as codebases grow. CodeOCR explores representing code as rendered images instead, taking advantage of visual compression capabilities that text lacks. The research shows vision-language models can understand code images at up to 8x compression while maintaining performance. Tasks like clone detection actually improve slightly under compression, and syntax highlighting boosts code completion performance at 4x compression. The approach suggests image-based code representation could fundamentally change how we handle large-scale code understanding.

RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System

https://arxiv.org/abs/2602.02488 | GitHub

Traditional reinforcement learning keeps environments, policies, and reward models separate. RLAnything makes all three components adapt together through closed-loop optimization. The policy trains on combined step-wise and outcome signals, while the reward model jointly optimizes through consistency feedback. Environment tasks automatically adjust difficulty based on critic feedback from both the policy and reward model. The system delivers substantial gains: 9.1% improvement on OSWorld for Qwen3-VL-8B-Thinking, and 18.7% and 11.9% boosts on AlfWorld and LiveBench for Qwen2.5-7B-Instruct.

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models

https://arxiv.org/abs/2602.07026

Despite progress in multimodal contrastive learning, a persistent geometric anomaly remains: embeddings of different modalities expressing identical semantics occupy systematically offset regions. This modality gap creates alignment challenges that existing approaches handle inefficiently. The research introduces Fixed-frame Modality Gap Theory, which decomposes the gap into stable biases and anisotropic residuals. ReAlign aligns text representations into image distribution space using statistics from unpaired data through anchor, trace, and centroid alignment. Building on this, ReVision enables MLLMs to learn visual representation distribution from unpaired text before visual instruction tuning, reducing dependence on expensive image-text pairs.

🎥 4 Videos

How to Make Claude Code Better Every Time You Use It (Full System)

Stop fighting with Claude Code and start building systematic workflows that compound over time. This comprehensive guide covers setting up persistent context files that Claude references across all sessions, structuring your codebase so Claude understands project architecture from the start, and creating reusable prompt patterns that extract better reasoning. You’ll learn how to build a project-specific knowledge base, manage multiple Claude sessions working on different parts of your code simultaneously without conflicts, and structure requests to get maintainable output instead of throwaway code. The system turns Claude Code from a one-off tool into a genuine development partner.

MoE, Visually Explained

Mixture-of-Experts architectures power many frontier models but remain conceptually opaque. This visual breakdown makes the core mechanics tangible: how router networks decide which experts process each token, why sparse activation improves efficiency without sacrificing performance, and what trade-offs exist between expert count and model capacity. The explanations focus on intuition over equations, making MoE accessible without dumbing down the actual complexity involved.

What is Indexing? Indexing Methods for Vector Retrieval

Vector databases require efficient retrieval mechanisms as collections scale. This overview covers indexing approaches from flat search through hierarchical navigable small worlds (HNSW) and inverted file indexes (IVF). You’ll understand when approximate nearest neighbor search makes sense versus exact matching, how different index types trade off speed against accuracy, and which methods work best for specific retrieval patterns.

Agents, Prompts, and RAG

Stanford’s practical breakdown of building reliable agents using retrieval-augmented generation cuts through the hype. The discussion examines when to use agents versus simpler prompt chains, how to structure RAG systems that agents can query effectively, and debugging approaches when agentic systems fail. The focus stays on production considerations: handling edge cases, managing token budgets, and building systems that degrade gracefully rather than failing catastrophically. Essential viewing for anyone moving beyond toy agent demos to production deployments.

📰 3 Curated Reads

Why Creativity Cannot Be Interpolated

https://archive.mlst.ai/paper/why-creativity-cannot-be-interpolated/

This exploration challenges the assumption that creative breakthroughs emerge from incremental improvements. The argument centers on fundamental limitations of interpolation-based learning: models trained to predict the next token in existing distributions struggle to generate genuinely novel ideas that exist outside their training manifold. The piece examines what this means for AI’s creative capabilities and why certain types of innovation may require fundamentally different approaches than current architectures support.

Prompt caching: 10x cheaper LLM tokens, but how?

https://ngrok.com/blog/prompt-caching/

Prompt caching stores processed prefixes so repeated context doesn’t get recomputed on every request. This technical breakdown explains how caching works under the hood: which parts of prompts get cached, how providers handle cache invalidation, and what billing implications exist. The post provides concrete strategies for structuring prompts to maximize cache hits, quantifies actual cost savings across different use patterns, and identifies scenarios where caching delivers minimal benefit.

Fine-tune MoE Models 12x Faster with Unsloth

https://unsloth.ai/docs/new/faster-moe

Training Mixture-of-Experts models typically requires massive compute due to their architecture. Unsloth achieves up to 12x speedups over Transformers v4 (and roughly 2x over the already-optimized Transformers v5) through custom Triton grouped-GEMM kernels and a Split LoRA approach that also cuts VRAM usage by over 35%. The documentation covers implementation details: how to integrate Unsloth into existing training pipelines, which MoE models are supported (Qwen3, DeepSeek R1/V3, GLM), and how the optimizations maintain full accuracy with zero approximation. Particularly valuable for teams fine-tuning large MoE models on consumer hardware.

🛠 2 Tools & Repos

Langextract

https://github.com/google/langextract

Extracting structured information from unstructured text typically requires custom pipelines for every new domain. Langextract is a Gemini-powered Python library that handles this through LLM-based extraction with precise source grounding, mapping every extracted entity back to its exact location in the source text. Define your extraction schema through a few examples and the tool adapts without fine-tuning, handling long documents through optimized chunking and parallel processing. Particularly useful for domains like clinical notes, radiology reports, and research literature where traceability between extracted data and source material is critical.

Deepeval

https://github.com/confident-ai/deepeval

LLM evaluation needs systematic frameworks rather than ad-hoc testing. Deepeval provides metrics for measuring answer relevancy, factual consistency, and hallucination rates across different model outputs. The framework supports both rule-based and model-based evaluation, enables A/B testing across prompt variations, and tracks performance degradation over time. Particularly useful for teams building production LLM applications that need reliable quality metrics.

🎓 1 Pick of the Week

Paper Banana

https://dwzhu-pku.github.io/PaperBanana/

Google’s Paper Banana went viral for good reason: it fundamentally changes how researchers produce academic illustrations. Instead of spending hours manually crafting methodology diagrams and statistical plots, Paper Banana orchestrates five specialized AI agents (Retriever, Planner, Stylist, Visualizer, and Critic) to generate publication-ready visuals from paper text. The system handles everything from architecture diagrams to data visualizations, with the Critic agent running multiple refinement rounds to catch factual errors and visual glitches. In blind human evaluation, its outputs achieved a 72.7% win rate against baseline AI models. For researchers drowning in illustration work or trying to produce consistent, high-quality figures across papers, this approach delivers production value that manual tools struggle to match. The viral response reflects how badly the research community needed a better way to handle the illustration burden.

Thanks for reading The Tokenizer! If you found something useful here, share it with someone who might benefit. And if you want more curated insights like this, consider subscribing to Gradient Ascent.

New here?

The Tokenizer is my resource-focused newsletter edition where I curate the best papers, videos, articles, tools, and learning resources from across the AI landscape. Consider it your weekly dose of everything you need to stay ahead in machine learning.

I’m teaching ML & Generative AI System Design on Feb 28th / March 1st with Packt.

Register Today

Gradient Ascent Special: Use code FLASH40 for 40% off

We’ll cover the core system design principles for building solid AI products: making systems reliable, measuring what matters, and designing architectures that work in production.

Through live discussions, guided exercises, and team-based design sprints, you’ll practice solving system-level AI problems and walk away with frameworks you can apply immediately at work.

What topics/problems would you most want covered in a system design workshop? Drop a comment or DM me.

TL;DR

What caught my attention this week:

📄 Papers: Vision encoders with causal reasoning for document understanding, open-source world simulators rivaling closed systems, plus advances in mathematical reasoning and multimodal scientific models

🎥 Videos: Pydantic fundamentals for ML engineers, building agent frameworks from scratch, production AI coding workflows, and understanding diffusion versus flow matching

📰 Reads: Reinforcement learning for continual LLM adaptation, preparing for ML interviews beyond just attention mechanisms, and DeepSeek’s latest architectural innovations

🛠 Tools: Curated resources for agentic reasoning research and comprehensive AI tool directories

🎓 Learning: Stanford’s updated large language model course covering recent advances

📄 5 Papers

DeepSeek-OCR 2: Visual Causal Flow

https://arxiv.org/abs/2601.20552 | GitHub

Instead of processing images in rigid raster-scan order, DeepSeek-OCR 2 introduces DeepEncoder V2 that mimics how humans actually read documents. The encoder uses causal attention to dynamically reorder visual tokens based on semantic content, determining whether to scan titles first, process tables column-by-column, or navigate multi-column layouts intelligently. By replacing CLIP with Qwen2-0.5B and implementing learnable queries with causal flow, the 3B parameter model achieves 91.09% on OmniDocBench while maintaining 256-1120 token efficiency. Reading order edit distance dropped from 0.085 to 0.057, proving the system genuinely understands logical document structure rather than just memorizing patterns.

Advancing Open-source World Models

https://arxiv.org/abs/2601.20540 | GitHub

Ant Group’s LingBot-World delivers minute-long consistent video generation at 16 FPS with under 1-second interaction latency, positioning open-source world models competitively against closed systems. The system maintains high-fidelity dynamics across photorealistic, scientific, and stylized environments through a multi-stage training pipeline combining web videos with Unreal Engine synthetic data. Users control camera perspectives and environmental conditions in real-time while the model preserves spatial consistency across 961 frames. The hybrid data engine with hierarchical captioning separates motion control from static scene generation, addressing the training data bottleneck that typically limits world model development.

Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation

https://arxiv.org/abs/2601.20614 | GitHub

Group Relative Policy Optimization suffers from an implicit imbalance where harder questions receive smaller policy updates, limiting capability development where it matters most. MathForge addresses this through Difficulty-Aware GRPO, which balances group advantage estimation by question difficulty and prioritizes harder problems through difficulty-aware weighting. The framework’s Multi-Aspect Question Reformulation strategy systematically increases question difficulty across multiple dimensions while maintaining gold answers, creating training data that pushes model boundaries. The synergistic loop of MQR expanding the data frontier and DGPO effectively learning from augmented data produces significant improvements across mathematical reasoning benchmarks.

Innovator-VL: A Multimodal Large Language Model for Scientific Discovery

https://arxiv.org/abs/2601.19325 | GitHub

Scientific multimodal models typically require massive domain-specific pretraining, but Innovator-VL demonstrates strong performance using fewer than five million curated samples without large-scale pretraining. The fully transparent training pipeline covers data collection, cleaning, preprocessing, supervised fine-tuning, and reinforcement learning with detailed optimization recipes, enabling systematic community extension. The model maintains competitive performance on general vision benchmarks while excelling at scientific tasks, indicating that scientific alignment integrates into unified models without compromising general-purpose capabilities. Principled data selection proves more effective than indiscriminate scaling for scientific reasoning.

Youtu-VL: Unleashing Visual Potential via Unified Vision-Language Supervision

https://arxiv.org/abs/2601.19798 | GitHub

Vision-language models exhibit a text-dominant optimization bias by treating visual signals as passive inputs rather than supervisory targets. Youtu-VL shifts to the Vision-Language Unified Autoregressive Supervision paradigm, integrating visual tokens directly into the prediction stream and applying unified autoregressive supervision to both visual details and linguistic content. This “vision-as-target” approach fundamentally changes optimization from treating vision as conditional input to making it a prediction objective. The framework extends to vision-centric tasks without requiring task-specific architectural additions, establishing foundations for comprehensive generalist visual agents.

🎥 4 Videos

Pydantic Crash Course

Dave Ebbelaar walks through Pydantic’s data validation and settings management capabilities essential for ML engineers working with LLMs. The tutorial covers defining models with type hints, validation logic, and configuration management patterns that ensure data integrity in production AI systems. Understanding Pydantic proves critical when building structured outputs from language models or managing complex application configurations where type safety prevents runtime errors.

Building Mini ClawdBot from Scratch

Vizuara demonstrates constructing an agent like Moltbot (ClawdBot, or whatever it's called now) without relying on existing libraries, revealing the actual mechanics behind agent architectures. By building from first principles, you understand tool integration, state management, and decision-making loops that frameworks abstract away. This approach proves valuable when debugging production agent systems or architecting custom solutions that don't fit standard framework patterns.

The Senior Engineer’s Guide to AI Coding

This How I AI episode examines Claude Code workflows and architectural decisions that separate effective AI-assisted development from mere prompt engineering. It addresses code review practices, testing strategies, and integration patterns when collaborating with AI coding assistants.

Flow Matching vs Diffusion Side By Side

Letitia compares flow matching and diffusion approaches for generative modeling, clarifying when each technique provides advantages. The visual comparison helps understand why flow matching sometimes offers training efficiency benefits over traditional diffusion while maintaining generation quality. Grasping these trade-offs matters when selecting architectures for specific generative modeling tasks.

📰 3 Curated Reads

Continual Learning with RL for LLMs

Deep (Learning) Focus

Continual Learning with RL for LLMs

Continual learning, which refers to the ability of an AI model to learn from new tasks and data over time, has become a popular topic in the discussion of Artificial General Intelligence (AGI). Put simply, general intelligence should be adaptable, which has led some to believe that continual learning abilities are a prerequisite f…

Read more

6 months ago · 93 likes · 3 comments · Cameron R. Wolfe, Ph.D.

Cameron Wolfe explores why Reinforcement Learning (RL) is naturally more robust than Supervised Fine-Tuning (SFT) for continual learning in LLMs. While traditional methods like replay buffers and regularization remain relevant, recent studies suggest that RL’s on-policy nature minimizes the distributional shifts that cause catastrophic forgetting.

Is Attention All You Need for ML Interviews?

https://medium.com/@maxwbuckley/is-attention-all-you-need-to-prepare-for-ml-interviews-830742f6d2ba

Max Buckley shares some quick tips to master the transformer architecture and its implementation ahead of ML interviews. Worth checking out to know the details of the bedrock of modern AI in case you’re preparing for interviews.

DeepSeek Drops Yet Another

WHERE MACHINES THINK

DeepSeek Drops Yet Another Architectural Innovation, Opening A New Front for Scaling Up LLMs

Some large language models can memorize entire books and regurgitate them almost verbatim. In one study, researchers successfully prompted an LLM to spit out, for example, a nearly complete Harry Potter and the Sorcerer’s Stone. While this raises huge concerns about large-scale copyright violations, it's also true that we simultaneously want these model…

Read more

6 months ago · 58 likes · 8 comments · Anil Ananthaswamy

Anil Ananthaswamy analyzes DeepSeek’s pattern of releasing architectural innovations that challenge conventional wisdom about model scaling and design. The examination covers their mixture-of-experts implementations, efficiency improvements, and open-source strategy that accelerates community progress. Tracking DeepSeek’s releases provides insights into architectural directions gaining traction in production systems.

🛠 2 Tools & Repos

Awesome Agentic Reasoning

https://github.com/weitianxin/Awesome-Agentic-Reasoning

This curated collection organizes research papers, codebases, and benchmarks focused on agentic reasoning capabilities in AI systems. Instead of accumulating every tangentially related work, the repository maintains editorial standards around core reasoning techniques like planning, reflection, and tool use. The structured organization helps researchers quickly locate relevant work when investigating specific reasoning approaches or comparing methodologies across different agent architectures.

GoodAI List

https://goodailist.com/repos

Chip Huyen curates AI tools and repositories with clear judgment about practical utility versus hype. The directory covers libraries, frameworks, and resources across the ML stack with emphasis on production readiness and actual adoption. Unlike comprehensive but overwhelming awesome lists, this maintains focus on tools demonstrating real-world value in AI development workflows.

🎓 1 Pick of the Week

Stanford’s LLM Course

https://youtube.com/playlist?list=PLoROMvodv4rObv1FMizXqumgVVdzX4_05&si=bqJCbjmvpCx-1_51

Stanford’s curated playlist brings together lectures from multiple courses covering large language models, including CS224N (NLP with Deep Learning), CS25 (Transformers United), and CME 295 (Transformers & Large Language Models). The collection spans foundational concepts in transformer architectures and training methodologies alongside cutting-edge developments in reasoning models, multimodal systems, and efficient inference techniques.

Thanks for reading The Tokenizer! If you found something useful here, share it with someone who might benefit. And if you want more curated insights like this, consider subscribing to Gradient Ascent.