Building a Resilient AI Agent Back Office with the OpenAI Agents SDK

# How to Build a Business Back Office That Runs on Agents with GPT-6 Astra (Complete Guide)

I will break down exactly how to build an agent desk that runs the repetitive half of a business on its own, and more importantly, how to build it so it survives the part nobody writes about.

Let's get straight to it.

Every guide you have read about automating a business with agents describes the happy path. Prompt goes in, tool gets called, work gets done.

Here is the number that guide left out.

Tool calling, the mechanism by which every agent in every one of those guides actually touches a real system, fails between 3% and 15% of the time in production. Not in bad systems. In well engineered ones.

That is the whole article. Not how to start an agent. How to build one that is still correct after the 3 to 15 percent.

## 1. The numbers first

Read the last two lines together. The model got dramatically better at the work, and OpenAI killed the no-code builder that was supposed to make the work easy. The SDK survived. That tells you where to build.

## 2. What an agent desk actually automates

Not "your business." Four specific shapes of work, and they are the ones where a 3 to 15 percent failure rate is survivable because a human sees the output before it matters.

Intake and triage. Inbound email, tickets, forms, invoices. The agent reads, classifies, extracts structured fields, routes. High volume, low individual stakes, and every output is a row a person can scan.

Document production. Contracts from templates, reports from data, onboarding packets. Astra creates documents, spreadsheets and presentations natively now, which removes the layer of glue code this used to need.

Reconciliation. Match what the CRM says against what the invoice says against what the bank says. Pure comparison work, and the agent's job is to surface disagreement, not resolve it.

Research and prep. Pull everything known about an account before a call. Nobody dies if it misses something, and the person reading it knows to check.

The shape these share is the shape you should look for: the output is reviewed before it is irreversible. Automation past that line is a different project with a different risk profile, and most of the guides that promise "automate 80% of your business" quietly cross it.

## 3. The primitives that exist for the failure rate

The OpenAI Agents SDK is deliberately small. Three primitives:

- Agents are models with instructions and tools, running a built in loop until the task completes

- Handoffs let one agent delegate to another

- Guardrails validate inputs and outputs, running in parallel with execution and failing fast when a check does not pass

That third one is the one this article is about, and it is the one most builds skip.

Around those three sit the pieces that matter for a real desk:

Install is one line:

Hello world is five:

That is the part every guide shows you. Now the part they do not.

## 4. Sandbox agents, and why permissions are the actual feature

The SDK's sandbox agents run a specialist inside a real isolated workspace, with a manifest that declares its files, a client that hosts it, and capabilities that declare what it can reach: filesystem, shell, memory, skills, compaction. There is a dedicated permissions module in the API reference.

Read that as a sentence: the SDK ships a mechanism for declaring what an agent is not allowed to touch, and it is the least discussed part of the documentation.

That is not a coincidence. It is the boring half. It is also the half that decides whether your 3 to 15 percent failure rate produces a wrong row in a spreadsheet or a deleted production table.

The docs are explicit about when to reach for this: use the Agents SDK when you need a real workspace or resumable execution. Use the Responses API directly when you want to own the loop yourself and the work is short lived.

You do not pick one globally. Most real applications use the SDK for the managed workflows and drop to the Responses API for the low level paths.

## 5. Why GPT-6 Astra changes the economics, and the cliff inside it

Astra shipped September 3, 2026. Four things matter for a back office desk.

It uses computers directly. OpenAI's own line is that anything you can do on a computer, Astra can do for you. For a back office that means the agent opens the CRM, reads the invoice PDF, fills the form, without a custom integration per system.

It holds 1,050,000 tokens. A full day of tickets, the whole contract, the entire account history, at once, not summarized.

It runs long. Multi step work over hours without losing the thread, which is what a reconciliation across three systems actually is.

The gap over the previous model is not incremental. 88.0% of tasks solved on the first attempt against 55.9% for GPT-5.6 Sol. Within four attempts, 99.2% against 68.7%.

Now the part that will show up on your invoice and not in the announcement.

Past 272,000 input tokens, pricing changes for the entire request. Input and cache rates go to 2x, output to 1.5x. Not for the tokens above the threshold. For the whole thing.

Two thousand extra input tokens nearly doubled the bill. A desk that naively stuffs "the whole account history" into every call crosses that line constantly and never sees why the monthly bill looks wrong.

The fix is not clever. Keep the working context under the threshold and let sessions carry what does not need to be in the prompt. That is what the sessions layer is for, and it is cheaper than the context window it replaces.

## 6. The eight agents

Each one owns a job. The point of the split is not capability, it is that a failure is attributable to a seat.

The rule that makes this work, and the one worth writing on the wall: only one seat writes. Seven agents produce candidates. One agent commits. Every irreversible action funnels through a single place you can audit, rate limit, and turn off.

Structured output is not optional here. OpenAI's own guidance is explicit that when downstream code needs typed data, use the output type. A drafting agent that returns prose to a filing agent that expects fields is the 3 to 15 percent failure rate wearing a different hat.

Note what the instructions spend their words on. Not on doing the job well. On what not to do when the job cannot be done.

## 7. The eight step build

Step 1: Install and set one agent working.

Start with one agent. The docs are unusually direct about this: split only when ownership, tools, approval rules, model choice, or trace clarity actually demand it. Most desks that fail were eight agents on day one.

Step 2: Give it structured output before you give it tools.

A tool that receives a string when it expected an integer fails at the worst possible moment. Define the Pydantic model first, make one agent fill it reliably, then move on.

Step 3: Add the guardrail that rejects your own agent.

Guardrails run in parallel with execution and fail fast. That is the design. A check that runs after the write has already happened is a log entry, not a guardrail.

Step 4: Put the writing agent in a sandbox with declared permissions.

The filing agent is the only one with write access, so it is the only one that needs a real workspace, and it is the one whose capabilities you declare explicitly. Filesystem scope, shell access, and what it may reach are manifest decisions, not runtime ones.

Step 5: Wire sessions instead of stuffing context.

This is the 272K threshold discipline from section 5, expressed as an architecture instead of a warning.

Step 6: Turn tracing on and actually read it.

Tracing is built into the SDK. It is also the thing that separates "the agent did something wrong" from "the extractor returned null and the drafter invented a value." Without it, every failure looks like the model being dumb, and you will respond by editing prompts, which almost never fixes it.

Step 7: Make the run survive a crash.

Two options, and they solve different halves.

Sandbox sessions in the SDK are resumable, which covers a run interrupted inside its own workspace.

For orchestration level durability, the Temporal integration runs each agent invocation as an activity inside a workflow, so a crashed process picks up where it left off instead of re-running and re-billing the whole task. On a long reconciliation that is the difference between losing four minutes and losing the token spend for the entire job.

Step 8: Draw the boundary, then check it.

Every seat produces work. One seat commits it. The commit is where you want a gate that is not an instruction in a prompt, because a prompt is a request and a gate is a rule.

For the code path specifically, this is what chalkline does: it reads the patch an agent produced and refuses it if the agent changed something its charter did not allow. Same principle applies to any write path. Declare what the seat may touch. Check it mechanically. Refuse the rest.

## 8. The six mistakes that kill these builds quietly

None of these announce themselves. They compound.

Context window as a dump instead of working memory. Every token you carry is a token you pay for and a token the model has to read past. Section 5 is what this costs.

Overengineered architecture before the problem demands it. Eight agents on day one, three of which exist because the diagram looked better with them.

Agents where a deterministic workflow does the job. If the steps are fixed and the branches are known, write the function. An agent is for when the path is not known in advance, and paying model prices for a decision tree is a choice, not a requirement.

Brittle output parsing. Fixed by structured output, ignored constantly.

Reaction instead of planning in the tool loop. The loop calls a tool, sees the result, calls another. No model of what it is trying to accomplish. This is why steps get skipped and tools get called out of order.

No evaluation from day one. Degradation stays invisible. You find out from a customer.

That last one deserves its own note now that OpenAI wound down the Evals product on June 3, 2026. The measurement layer is yours to build or source. That the vendor stopped selling it does not mean you stopped needing it.

## 9. Where this genuinely does not work

Anything irreversible without review. Sending money, signing, deleting, publishing. Not because the agent cannot do it, but because a 3 to 15 percent failure rate on an irreversible action is a different category of problem than a 3 to 15 percent failure rate on a draft.

Judgment where being wrong is expensive and being confident is easy. Legal interpretation, medical, anything where a fluent wrong answer reads exactly like a right one to the person receiving it.

Systems with no API and no stable interface. Astra's computer use covers more than an integration would, but a UI that changes weekly will break a browser agent as reliably as it breaks a scraper.

Prompt injection is not solved. An 11.2% success rate in production, improved from 23.6%, is a real improvement and still a number you have to design around. Astra is documented as significantly more robust than GPT-5.6 Sol here. It is not immune, and any agent that reads untrusted inbound text, which is exactly what an intake agent does, is in scope.

## 10. What I would cut if I started this over

Three things I would not build again, and the reason each one felt necessary at the time.

The classifier as a separate agent. It reads a ticket and returns a type. That is a job for a model call, not for a seat with its own instructions, its own trace, and its own failure mode. Splitting it out made the architecture diagram cleaner and made the actual run slower and harder to debug. If a seat has no tools and no branch, it is a function call wearing a costume.

Retry as the answer to everything. The first version retried any failed tool call three times. That turns a 5% failure rate into a 0.0125% failure rate on paper, and on a write path it turns one duplicate invoice into three. Retry is correct for reads. On anything that commits, it needs to be the last thing you add, not the first, and it needs an idempotency key before it needs a retry count.

A confidence score I did not calibrate. The extractor returns a number between 0 and 1 and the guardrail rejects anything under 0.7. I never checked what 0.7 meant. When I finally sampled a hundred outputs against the source documents, the model was roughly as accurate at 0.65 as at 0.9, which means the threshold was doing nothing except rejecting work at random. A number you have not measured against reality is a decoration that looks like a control.

The pattern in all three is the same. Each one made the system feel more careful without making it more correct, and that feeling is expensive because it stops you looking for the thing that would have worked.

## The point

The reason most agent desks fail is not the model, and it has not been the model for a while now. It is that the system around the model was built for the run where everything works.

The three to fifteen percent is not an edge case to handle later. It is the design constraint. Build the guardrail before the tool, the boundary before the write, the trace before the scale, and the desk survives contact with real work.

Skip them and you get what everyone gets: a demo that was impressive in March and quietly stopped being trusted by June.

Sources, for anyone who wants to check the numbers or pull a quote directly. The SDK primitives, sandbox permissions, sessions, and the Responses-API-versus-SDK guidance are from OpenAI's official Agents SDK documentation at openai.github.io/openai-agents-python and the repository at github.com/openai/openai-agents-python. GPT-6 Astra pricing, the 272K threshold, context size, and the first-attempt figures are from OpenAI's model documentation and OpenRouter's published rates. The Agent Builder and Evals wind-down date is stated on OpenAI's own AgentKit page. The 3-15% tool call failure figure and the ChatDev correctness number come from published production write-ups; the 11.2% prompt injection figure is from Anthropic's research as cited in the same. The Temporal durable execution pattern is documented at temporal.io. The six failure modes are drawn from Decoding AI's production agent guide. For hands-on material, the Agents Towards Production repository collects 25 tutorials across orchestration, observability, memory and deployment.

Figures reflect what was published as of September 2026. Verify current pricing and model behaviour before building on either.

If you want more breakdowns like this, I post one every couple of days on Telegram and X. Both free.

X - https://x.com/gippp69

Telegram - https://t.me/GipArcAI