Shift Left, Again: Notes From Our First Week With Jev

Roughly a week ago, the X feed exploded with TypeSafe AI’s launch of Jev, the first System One model. If you haven’t heard about Jev yet, let me start by applauding your social media hygiene. You can think of Jev as an extremely fast, extremely cheap decision model: it gave up the ability to generate a stream of tokens, but in return can answer yes/no questions or make typed classification decisions at ridiculous speed.

It took me a day to actually notice the launch, but once I watched the demo, my mind was fully consumed by the seemingly endless possibilities of integrating the new tool into every layer of our system. X’s echo chamber kept pushing more and more cool examples of things you can build with Jev, from video-game-playing AI and smart clipboards to very fast browser use. It inspired us to look at our day-to-day problems for more places where the new toy might fit.

Broadly speaking, the excitement comes from both the cost and latency promises. With the cost per unit of intelligence rapidly going down [source needed], the price is appealing but may not be a fundamental property of the tool. The latency- on the order of 100 milliseconds - however, makes Jev feel like a genuinely new building block for AI-driven applications and agentic systems.

In this post, we’ll share some of the things we liked and have started adopting in our workflows and product, along with the properties that made them good candidates for Jev-based workflows.

[Shifting Left the Agentic R&D]

In a recent post, our co-founder Yair Weinberger described how we try to prevent coding agents from letting slop into the codebase.  

Coding agents made it extremely easy to write custom deterministic lint rules to enforce all sorts of coding standards during development and provide the agents with a tight feedback loop. 

At Reindeer we’ve invested in a large set of home grown linter rules, along with popular industry tools (Ruff and Biome) to encode the use of best practices, as well as local coding preferences. Anything that can be represented as code - from using a specific logger to having exactly one implementation of a specific interface, becomes a custom linter add-on.

Unfortunately, some coding quality rules can’t easily be expressed as linting rules, unless you are willing to make the structure of your code very awkward (e.g. It is not trivial to codify a rule like “Services should own business decisions, and repositories should own persistence”). A common approach many teams land on is a set of semantic soft rules, encoded either in some AGENTS.md file or a custom documentation, and ask an LLM reviewer to evaluate them in CI pipelines

This “Agentic Code Review” seems to be the standard industry recipe, but anyone who had gone through the process is also familiar with some of the paint points:  Review agents often miss critical context from the production environments to make the right judgement calls, they often amplify the work by reporting low value findings, and the cost of running them on frontier models is not negligible.

The less discussed cost is what late feedback does to the coding agent.  Dex Horthy coined the nice term - “the dumb zone”: by the time the Agent consumed large part of its’ context, the quality of its decisions begins to degrade just as review asks it to revisit earlier work

What if we could shift that semantic judgment earlier? Jev's latency and cost profile makes it feasible to evaluate selected rules while the agent is still working on the PR. We’ve released the Pi Jev Guard - an extension that checks proposed writes against repository instructions before Pi writes them to the file. An agent producing slop can course-correct with a gentle slap on the wrist, before it faces a harsh code review.. 

This is still early days for this specific approach, and we’re still evaluating whether the Write tool is the right integration point (vs. a pre-commit hook, or just a very cheap CI pipeline). 

Having said that, there is a wider takeaway here - ultra fast semantic code quality enforcement has become possible, and adopting Jev into the SDLC is a very clean first use-case.

[Guardrails in the Hot Path] 

Reindeer agents execute long operational workflows, which often interact with critical back-office systems and access sensitive data. Performing this task requires trust, and we bake guardrails into the execution, across multiple layers.  

If you have used the Auto Mode feature in the leading coding harnesses, you will recognize the technique of running each Agentic action through an ensemble of classifiers. These are relatively fast, limited context LLMs that perform adversarial checks against the main agent, and review the command against distinct criteria.  In Reindeer, we have a few layers of such checks, which verify adherence to the official user policy configuration (e.g. “is this action aligned with the user defined permissions?”), as well as malicious attempts (e.g. “is the agent trying to escape the Reindeer sandbox”?). 

It is important to highlight the cons of the approach. Relying on non-deterministic LLM calls to enforce compliance cannot be the only security mechanism, but it also introduces interesting performance implications and failure modes.  

First, the LLMs are encouraged to be incredibly conservative, and err on the side of caution. It is often advisable to block ambiguous commands with clear warning to the Agent, which will then course-correct, than to allow a divergence from the well-lit path. Second, when you consider an agentic workflow that spans 200 or 300 turns (common in the complex operational workflows we run), adding another set of LLM-bound calls in the hot loop can easily accumulate a lot of latency. Hence, when Jev was released, we were curious to check whether it can serve as a competitor to our existing models.

We compared Jev with the two models already serving this role in our stack: Gemini 3.1 Flash-Lite and GPT-5.6 Luna.

The general-purpose models received the proposed command and the relevant policy, and returned a structured `allow` or `block` verdict. 

For Jev, we translated the policy into bounded yes/no questions and blocked the action when the probability of a blocking condition crossed its calibrated threshold. We selected the thresholds on a small calibration set, then evaluated the models on almost 1000 historical examples, using known regressions and judgments from a stronger frontier model (Claude Opus 5) as the reference.

We’ve measured the agreement with Opus as the proxy for accuracy.

The findings here are not surprising. Jev matched Luna at 97.26% agreement with the reference set, outperforming Gemini Flash Lite, while also substantially reducing the latency and the cost of the process.  

We can expect many one-shot LLM calls converting to TypeSafe classification decisions, as this seems to be the bread and butter for Jev.

[Bounded Answer Space] 

One of the first demos I came across after Jev launched was Jev Ultrafast by Browser Use. It completed a Google Flights search from Zürich to London in 7 seconds. I found it interesting because Jev was not used merely as a classifier. It was running repeatedly inside the browser loop and choosing the next action, driving the full scenario. 

Browser use looks like an open-ended problem, but the harness narrows each step considerably. It extracts the controls that are currently visible from the DOM, and gives Jev a small set of possible actions: click, type, select, scroll, wait, or stop. Jev selects the action and its target, and the harness validates and executes it. The action that requires text generation - typing - is offloaded into a small LLM. The page is then captured again and the same loop continues. 

I think this is the more general pattern behind the demo. Some workflows are long, but the decision at each step is still small. If the system can describe the current state, enumerate the legal moves, and verify the result, Jev can drive the sequence without running a general-purpose LLM on every turn. The benefits are compounding if the sequences are long, and the stakes are relatively low.

We started using the same structure in our Frontend QA pipeline. Jev moves through the app, using the actions available on each page and tries interactions that may expose broken flows or missing behavior. Previously, running the same flow with LLMs would still cost a dollar and a couple of minutes. The current pipeline, however, quotes in less than one cent and is bound mostly by the response time of the backend, rather than by the LLM, making the shift left feasible once more.

‍These are the places we’ve tried Jev so far. I suspect there are more interesting uses we haven’t found yet. If you’ve put it into a real workflow, I’d love to hear what worked, what broke, and whether the numbers held up. And if figuring out these kinds of systems sounds like your kind of work, come talk to us at Reindeer.