16 months of outcome-first engineering.
The world’s largest organizations run their talent on Eightfold. Fortune 1000 companies trust our platform with business-critical processes, hiring, internal mobility, and workforce planning, for millions of candidates and employees who never get a second first impression. When the stakes are that high, “move fast and break things” is disqualifying. We cannot afford to be fast and buggy. We have to be fast and better, at the same time, in every dimension that matters: security, capability, AI quality, and governance.
That is the real test for AI in engineering. It is easy to use AI to produce more. The hard part, and the only part that counts for customers like ours, is using it to ship faster while raising the bar on quality across the board. Over the past 16 months, that is exactly what we set out to prove, and to measure. The result: median epic cycle time down by about two-thirds, 86% of security false-positive noise filtered before an engineer sees it, support root-cause accuracy up from 60–65% to 72–75%, and first responses that now land in minutes instead of hours, all with a person accountable at every step, and AI acting on its own only where it has earned that trust.
None of that came from doing the obvious thing, which is to count how much AI everyone uses and call it progress. In fact, getting these results meant deliberately refusing the industry’s favorite new shortcut.
Four rules for putting AI to work in engineering while keeping people in charge.
At Eightfold, we don’t rank engineers by how much AI they use. Instead, we give AI specific, countable jobs, keep a person accountable at every step, and measure what it finished and how good it was. Over 16 months that approach cut median epic cycle time by about two-thirds without letting AI-inflated token numbers stand in for progress.
Here’s how we got there. We start with the trap we chose to avoid, token-maxxing, and the four rules we built instead. Then we follow those rules through five areas of our work: plugins and skills that share team know-how, Helixa for support tickets, an on-call agent for production alerts, a Dev agent pipeline that takes epics to merged code, and a security agent that cuts through alert noise. Each section covers the same four things: the problem, what the agent does, where a person stays accountable, and what we measure.
The trap we avoided: token-maxxing.
Over the past year, a habit spread across the industry: counting AI usage as if it were output. Teams built leaderboards ranking engineers by tokens consumed, and some tied AI usage to performance reviews. At one large tech company, a leaderboard ranking tens of thousands of employees by token use was shut down within days after people left agents running to climb it. Researchers found coding agents raised lines of code by 741%, while releases rose only 20%.
Industry benchmarks tell the same story (DX, Q2 2026). Developers save 4–6 hours a week with AI, yet the share of time spent on new features stays flat at about 57%. Commits rise by up to 180%, but production releases rise only 20–30%. And developer experience scores dipped as code review became the new bottleneck. More activity, it turns out, is not more delivery.
Our own data shows the same trap. Net lines of code per engineer grew 4.2x since April 2025, but much of that is bulk commits and generated code. If we had celebrated that number, we would have celebrated noise.
Tokens are a cost, not a result. The question we ask instead: what did the AI finish, how good was it, and what’s the outcome?
Four rules for agents we build
- Pick work you can count. Support tickets, alerts, pull requests, epics. If you can’t count it, you can’t tell whether AI helped.
- One agent, one job. Each agent has a narrow role and clear limits. No general bot that does everything.
- A person stays accountable. AI does the legwork. People approve designs, merge code and handle the hard cases. AI acts on its own only where it has earned that trust.
- Measure outcomes per result. Cycle time saved, quality improved. Never tokens per engineer.
AI plugins and skills: share know-how.
The problem. When every engineer writes their own prompts, good practices stay in one person’s head, and nobody knows what anything costs.
What we did. We package team know-how as AI plugins and skills, published through an internal marketplace. Every run logs its key steps, so we can see which skills each team uses and where they fail.
- Write once, improve for everyone. A skill is versioned and updated in one place, instead of living as copies of someone’s prompt that drift apart. A new version reaches everyone automatically.
- Cost-aware by design. When a skill uses too many tokens, we move the predictable steps into plain scripts and use the model only where it adds judgment.
Two plugins show how far this reaches beyond engineering:
The metrics we watch are completion rate and outcomes, not how many tokens each team burns.
Home grown support agent- Helixa : a root-cause analysis with auto reply capability for support tickets.
The problem. Before anyone can answer a support ticket, someone has to work out what went wrong: check the data and configuration, read logs, trace the code. That investigation is where most support engineering time goes.
What the agent does. Helixa investigates incoming tickets and writes a root-cause analysis with a proposed fix. For tickets that have >95% confidence, Helixa replies to the customer directly. For the rest, support engineers review the analysis and use it to respond. Each analysis is rated against how the case was actually resolved; 72–75% are now rated correct or likely correct, up from 59–65% when we first wrote about Helixa in January.
CRM support case data · root-cause analyses on closed support incidents, rated against how each case was resolved, Oct 2025 – Sep 2026
Earn trust first, then automate. The agent was only allowed to reply to customers directly after months of data demonstrated it was reliably correct, and only on high-confidence tickets. Everything else still goes to an engineer with the analysis attached. A daily process grades every auto-reply against how the case ended and turns repeat mistakes into proposed fixes. Of auto-replies with a clear customer outcome, 82% were on track: either solved outright or in the right direction with a person adding the specifics.
The results: when the AI replied, the median time to first response dropped from about 7 hours to 24 minutes, which is roughly 18x faster (the AI handles more routine cases, so part of the gap reflects case type). Auto-replies alone have saved thousands of analyst hours since April.
The lesson. Measuring outcomes showed us something a usage dashboard never would: faster first answers don’t always mean faster resolution. When a person had to rework the AI’s reply, the case waited longer in the hand-off. So our next fix isn’t a bigger model; it’s routing those cases to a person immediately.
The on-call agent: an answer before you open your laptop.
The problem. Traditional alerts cost an engineer 30 minutes to an hour of context-switching before they even know what broke. The knowledge to fix it often lives in a few people’s heads.
What the agent does. When an alert fires, the agent picks it up, follows that team’s playbook, runs the queries itself, pulls logs across every region, and posts its findings in the alert thread. The on-call engineer starts from a likely cause and next step, but still makes the call.
It runs on every engineering team’s rotation, and on our busiest rotations it handles what would be hours of investigation every day. Teams extend it by adding a markdown playbook to the repo; once merged, the agent uses it on the next alert, with no deployment.
Monitor its budget like any production dependency. We measure our engineers on outcomes, not spend, but we still guard against runaway agent spend. Early on, our AI credit balance ran out. The agent stopped without saying so, and production alerts went untriaged for over an hour. Since then we’ve added health checks, pre-run budget checks and low-balance alerts. If an agent is part of production, its budget needs monitoring like the rest of production.
The Dev agent: from epic to merged PR.
Engineering output is limited by how much time people have, not by what AI can do. Our Dev agent pipeline is a set of agents that take an epic in our issue tracking software from design to merged code, with a person approving at every step:
Two choices keep a person’s judgment at the center. First, the TL Agent offers 2–4 design options with their trade-offs, never a single answer. That way the reviewer has to weigh the options and pick one instead of rubber-stamping whatever the AI produces. Second, a separate agent grades each PR against the design’s acceptance criteria before any person spends time on it.
We’ve tested the full loop end to end by rebuilding one already-shipped feature from its epic.
The security agent: precision over volume.
Security scanners produce a lot of alerts, and most aren’t real risks. Engineers learn to ignore them, and real issues get lost. Our security agent reviews pull requests, independently verifies security fixes, and checks whether vulnerable code can actually be reached and exploited before an alert reaches a person. Every epic also gets an AI-generated threat model and an agentic penetration test before release.
86% of false-positive vulnerability noise is filtered out before a human ever looks at it.
The team also tests whether cheaper open-weight models can do parts of the job at the same quality. Costing less for the same result is a win, even if total usage looks smaller.