customers
Production stories, with the methodology attached.
Three engineering teams, three different workloads. Every number below states how it was measured — a metric without a denominator is marketing, and we'd rather show you traces.
Halyard
Fintech — expense management · Toronto, ON · 340 employees · 60-person engineering org
Halyard's support agents resolve 62% of tier-1 tickets end-to-end — and the traces are why
Halyard's support team was drowning in receipt disputes and card-limit questions — high-volume, policy-bound tickets that a well-guarded agent should handle. Their first agent, built in-house, worked in demos and fell apart in production: silent failures on tool timeouts, no way to see why a run went sideways, and a growing fear of what it might do to a customer's card limit.
They moved the agent onto Trellis in a two-week migration. Guardrails came first: a tool allowlist that keeps the agent away from write endpoints above $500, argument schemas on every refund call, and budget caps per run. Then the Friday trace review became ritual — the team reads a sample of failed and expensive runs, flags them into eval datasets, and prunes the workflow.
Six months in, the run is down from nine model calls to six, p95 latency halved, and the escalation path is clean: the 38% of tickets the agent can't finish arrive at a human with the full trace attached — context, attempted steps, and the exact policy that stopped it.
Stack: Support · Economy models · Standard guardrails · Full traces · ~180k runs/mo
“We didn't get to 62% by making the agent smarter. We got there by reading traces every Friday and deleting the steps that weren't earning their tokens.”
- 62%
- of tier-1 tickets resolved end-to-end
- 14s → 6s
- p95 run latency after trace-driven pruning
- 0
- out-of-policy refunds since enforce mode
measured: trailing 90 days; 'resolved' = customer confirms or no reopen within 7 days
measured: same ticket mix, measured across 40k runs before/after
measured: guardrail audit log, January–June 2026
Corelab Genomics
Biotech — genomics tooling · Cambridge, MA · 120 employees · 11-person ML platform team
Corelab's eval gate caught a citation-fabrication regression before a model upgrade shipped
Corelab runs research agents that read genomics literature and draft evidence summaries for scientists — work where a fabricated citation isn't an oops, it's a liability. Their eval discipline started after a near-miss: a summary citing a paper that didn't exist, caught by a reviewer a day before it went into a partner report.
Every flagged production trace now feeds a golden dataset — 2,400 cases and growing — graded by a calibrated LLM judge for citation accuracy plus deterministic checks on extracted values. The suite runs on every prompt change and every model candidate, gated in CI with `trellis eval --gate`.
In April, a frontier model upgrade looked like a free win: faster, cheaper, better on most suites. The gate failed it — citation fabrication rate rose from 0.8% to 4.1% on the golden set. The team shipped the upgrade three weeks later behind a revised retrieval step, with fabrication back under 1% and the diff to prove it.
Stack: Research · Frontier models · Strict guardrails · Full traces · ~9k runs/mo
“The candidate model was better on eleven of thirteen suites. The gate failed it anyway, because the two it regressed on were the two that matter: citation accuracy and dosage extraction. That one blocked deploy paid for the year.”
- 0.8% → 4.1%
- fabrication regression caught by the gate
- 2,400
- eval cases grown from flagged production traces
- 13
- eval suites gating every deploy
measured: citation-accuracy suite, 2,400-case golden set, candidate vs. baseline
measured: cumulative since September 2025
measured: CI config, June 2026
Bracken Freight
Logistics — freight forwarding · Memphis, TN · 900 employees · 45-person platform engineering group
Bracken automates 1.1M ops runs a month against its TMS — with guardrails blocking every out-of-policy write
Bracken's ops agents reconcile shipment records between customer EDI feeds and their transport-management system: correcting codes, filling missing weights, flagging genuine discrepancies to humans. Small tasks, brutal volume — around 1.1 million runs a month on economy models.
The engineering constraint was never intelligence; it was blast radius. Every write the agent makes goes through argument-schema validation and a tool allowlist that partitions 'safe corrections' from anything touching pricing, contracts, or customer master data. Strict guardrails screen every call, and forensic traces keep 90 days of full payloads for their auditors.
In Q1 2026 the guardrail layer blocked 1,142 write attempts that fell outside policy — most of them the model over-reaching on ambiguous records, exactly the failure mode everyone warned about. Each block became an eval case. Zero out-of-policy writes reached the TMS.
Stack: Ops · Economy models · Strict guardrails · Forensic traces · ~1.1M runs/mo
“The agent is allowed to fix a mislabeled shipment. It is not allowed to touch a rate table. The difference isn't a prompt — it's a schema and an allowlist the runtime enforces. That's the only reason legal signed off.”
- 1,142
- out-of-policy writes blocked in Q1
- 1.1M
- runs per month on economy models
- 97.4%
- of runs complete with no human touch
measured: guardrail enforcement log, Jan 1 – Mar 31, 2026
measured: average of trailing 3 months' invoices
measured: runs ending in 'reconciled' state / all runs, trailing 90 days
how we report
Rules for every case study on this page
- Every metric names its window, denominator, and source system.
- Before/after comparisons hold the workload mix constant or say they can't.
- Customers approve the numbers and the caveats — both ship or neither does.
- No composite "up to" figures, ever.
$ trellis estimate --honest
Want the same trace discipline on your fleet?
Estimate the cost first, then bring your gnarliest workflow to a demo. If a pipeline would serve you better than an agent, we'll say so in the first call.
All customers, people, and figures on this page are fictional, invented for this sample site.