Fast, lightweight semantic-decision models like Jev enable distributed micro-inference throughout an application rather than concentrating AI reasoning inside agent loops.
Adapted from @JoshARosen# System One Models: What Comes After Agentic Software TypeSafe released Jev this week, a model built for fast semantic decisions instead of generation or long-form reasoning. You give it state and typed questions, and it returns choices, scores, or probabilities. TypeSafe reports latency as low as 70ms and pricing low enough to invoke the model far more frequently than the frontier models we build around today. (https://typesafe.ai/) (https://typesafe.ai/blog/introducing-system-one-models-and-jev) Jev arrives at an interesting point in AI architecture. The agentic era is in full swing and was unlocked by models getting good enough at reasoning through multi-step problems and reliably calling tools. We’re all in the middle of reorganizing our applications around those capabilities by building long-horizon agent loops, harnesses, and software factories. But System One models are not designed for building agents. Their value may come from distributing small amounts of semantic intelligence throughout an application rather than concentrating it inside an agent. (https://typesafe.ai/blog/introducing-system-one-models-and-jev) This raises a much bigger question: what new classes of AI software will they make possible, and how will we build them? ## A New Micro-Inference Era? Most agent architectures concentrate non-determinism inside a loop. The surrounding system assembles context and exposes tools. The model interprets the situation, decides what to do next, takes an action, observes the result, and continues. This creates a fairly obvious architectural boundary. Deterministic software manages the environment around the agent while non-deterministic decisions happen inside the loop. A lot of recent agent infrastructure has focused on controlling this boundary through permissions, structured outputs, state, and deterministic orchestration. System One models allow inference to move across that boundary. Routing and validation can now contain semantic judgment. State transitions and dependency resolution can depend on model outputs. Instead of sending a substantial unit of work into a model loop, the application can invoke intelligence at individual and tiny decision points throughout the code. This gives us a different way to compose AI software. An agent packages intelligence into a worker that receives work and determines how to perform it. “Micro-inference” can more natively weave intelligence into the system around it. ## A Finer Deterministic Boundary AI builders have spent a lot of time deciding what belongs in code and what belongs in the model. Routing, permissions, retries, dependencies, and state transitions are increasingly moving into deterministic infrastructure. Models are used where the system needs interpretation or judgment. System One models push that separation down to a much finer level. An individual branch in application logic can depend on a semantic judgment or a validator can contain inference. You could also build state transitions that depend on a probability. That means the codebase itself may contain much more non-determinism. Rather than a clear boundary between your deterministic code and beefy model calls, we’ll start to see smaller model calls scattered throughout the codebase. At the system level, the opposite will be true. Applications will explicitly define more of their control flow and own more of their orchestration because they no longer need to hand a large decision surface to a reasoning model. For example, a model can determine whether a requirement is satisfied while software determines the consequences. In other words, more individual decisions can involve inference while more application behavior remains explicitly modeled in software. ## Reasoning Moves Into the Architecture Jev is designed around what TypeSafe calls System One tasks. It makes fast decisions rather than performing the slower deliberative reasoning we increasingly expect from frontier models. Using this model class broadly forces an architectural question about where the reasoning happens. Today we often give an entire semantic problem to a reasoning model and let it own decomposing the problem, taking actions, and providing a solution. A system built around cheap decision models would need to build that decomposition at a higher level in code. Instead of asking one model whether an implementation satisfies a spec, the application could break the spec into individual requirements and ask smaller models whether each one is satisfied. Code could then combine those answers, flag disagreements, and send the few unresolved requirements to a reasoning model. A reasoning model still has a role in this architecture. It can handle ambiguous cases, investigate missing evidence, or perform work that cannot be reduced to smaller judgments. It just no longer needs to own the entire computation. This creates another dimension for AI architecture. Builders can choose how much reasoning happens inside a frontier model and how much is expressed through the composition of software and smaller models. ## Need for Micro-Inference Infra Applications making thousands of small semantic decisions will need infrastructure built around a very different inference pattern. Calling model APIs directly from every branch in the application would quickly create problems around latency, cost, consistency, and observability. I expect libraries and runtimes to appear (ready, set, go) around this micro-inference layer. They can batch independent judgments against shared state, cache results when the inputs have not materially changed, and execute large groups of decisions in parallel. They can also maintain confidence thresholds and determine when a decision needs escalation. Model routing gets pushed down into this layer as well. A cheap decision model can handle routine judgments while uncertain cases move to stronger models. A reasoning model can become an escalation path for decisions that require decomposition or investigation. High-risk judgments can run through multiple models or deterministic checks before the application acts. The programming abstraction can move away from calling a specific model. Developers define semantic operations with typed inputs and outputs, while the runtime determines how those operations execute. ## General Semantic Models Enter the Control Loop Embedding inference inside application execution actually has a long history. Autonomous vehicles continuously run perception and prediction models while conventional software coordinates planning, safety constraints, and control. Waymo describes its current architecture as ML-primary, with heterogeneous neural networks running alongside conventional compute responsible for orchestration and other non-ML work. (https://waymo.com/blog/2026/08/look-under-our-trunk/) Those architectures have generally depended on specialized models. A perception model recognizes a particular class of objects. A fraud model predicts a specific kind of risk. Each model is built around a relatively narrow operation. Language models gave developers much broader semantic capabilities, but their latency and cost encouraged larger units of inference. We built chat applications around generation calls and agents around reasoning loops because those were natural ways to package expensive general-purpose intelligence. System One models bring general semantic judgment closer to the operating characteristics of embedded ML. A developer can define a new semantic decision through inputs and typed outputs rather than training a dedicated classifier for every decision. Architectures common in autonomous systems can therefore start showing up in ordinary business and developer applications. ## The Doom Demo Shows the New Execution Model TypeSafe demonstrated Jev playing Doom at roughly ten model calls per second. The application provides structured game state and repeatedly asks the model to choose actions. The interesting architectural property is the frequency at which a general semantic model can participate in the application’s control loop. (https://typesafe.ai/blog/introducing-system-one-models-and-jev) That pattern looks very different from an agent receiving a task and working until completion. The game continues to execute while inference repeatedly contributes small decisions. Intelligence is part of the execution path rather than a separate worker invoked to accomplish a unit of work. The same structure can appear in applications far removed from games. A development system can continuously evaluate which requirements and artifacts are affected by a change. A security system can interpret actions against intent and policy as they occur. A business application can continuously reconsider existing work as its underlying state changes. These systems may invoke agents when substantial work needs to happen. The decision that work needs to happen can come from a much finer layer of intelligence already running throughout the application. ## What Comes After Agentic Software The agentic era came from a change in what models could reliably do. Once models could reason across multiple steps and use tools, builders created an architecture around those capabilities. The agent loop became one of the defining abstractions of the current generation of AI software. System One models give us a different set of properties to design around. The resulting applications may contain far more model inference than today’s agentic systems while depending less on agents to control their behavior. Agents become one component inside a larger intelligent architecture rather than the architecture itself. Chat software was organized around generation. Agentic software was organized around reasoning and tool use. Models like Jev give builders another model primitive to organize software around, and we are only starting to see what that software looks like.
