Amazon asks its engineers to explain where AI does not help their work

On 27 October Amazon published a memo from Terk Terkowitz, vice-president and distinguished engineer for Worldwide Amazon Stores, explaining a revised role guideline for its software development engineers (SDEs). The memo, already shared with engineers across the company, tells them and their software development managers (SDMs) that the outcomes the role is judged on – impact on customers, the business and teams – have not changed, even as AI changes how the work gets done.

Amazon says that in some cases it has used input metrics such as token usage and adoption of AI tools, and throughput metrics such as code changes and reviews, because the outcomes it cares about “can be time delayed, difficult to measure, and hard to isolate from other trends”. The memo then says: “We are all aware of how deeply flawed these metrics are, primarily because they are disconnected from what matters.”

The memo comes after an episode recounted by Fortune: an unofficial internal AI leaderboard called Kirorank, which Amazon shut down in May after tokenmaxxing – running up AI usage to climb the rankings – raised concerns about unnecessary computing costs. The memo names the behavior directly: “tokenmaxxing is at odds with frugality”, and “measuring LOC is at odds with invent and simplify”, setting tokenmaxxing and the counting of lines of code (LOC) against two of Amazon’s leadership principles.

Amazon says it chose not to define good AI practice in the guideline itself, because any definition “will likely look antiquated by this time next year”, and anchored the role on outcomes instead. The memo around it still tells thousands of engineers how AI use will be read, counts judgment about where AI does not help as part of the job and keeps the usage numbers it calls flawed. For any employer that has started counting how much its staff use AI, Amazon has put in writing what those counts miss and why it will keep using them anyway.

Flawed metrics, kept anyway

The memo does not drop the numbers. It says the input metrics “correlate well with the outputs that matter” and that Amazon expects them to keep doing so “for some time”. It credits that to higher productivity and delivery on what it calls frontier teams, the heaviest AI adopters, and says teams and individuals “who are not adopting AI or increasing output metrics are falling behind”.

The memo recommends that engineers and their managers watch the 10th percentile of AI adoption, “finding those teams who aren’t embracing AI effectively and understanding why”, and the 90th, to see whether heavy users are “seeing delivery increases to match”. The second check tests the metric itself: a team at the top of the usage table with no matching gain in delivery is the extra-cost case the memo warns about.

Opting out, on the record

The memo’s most direct line on people concerns engineers for whom AI does not work. “If your work does not benefit from current AI tooling, that is a finding, not a failing,” it says. It asks them to “document why, share it with your community of peers, bring it to your manager, and work to find solutions”, and says an engineer who can explain where AI does not yet help is showing the judgment Amazon wants.

The same memo also tells engineers: “If you haven’t already, you should take the time to learn and apply AI into your daily work as soon as possible.” It adds: “We all need to embrace this change and take the time to learn, or we risk being left behind.” An engineer whose work does not benefit is expected to write down why, and the 10th-percentile check asks why the teams lowest on AI adoption are not embracing it effectively.

A hands-on test for Amazon’s leaders

The memo also sets a bar for leaders. “Sufficient understanding means you have used these tools on your production work, can describe a task where they helped and a task where they did not,” the memo says. Leaders are also expected to explain that call to their teams. Technical and organizational leaders are told their job includes making sure their team uses AI “to the best effect”, and that Amazon expects them to take “ownership of connecting the inputs and outputs in your team or organization”.

According to the memo, the leaders having the most impact are those spending the time to learn AI themselves and “making time for the people they lead to do the same”, and the returns on that investment have been “disproportionately large”. It gives no figure for those returns, and none for the productivity of its frontier teams.

It also names a drift Amazon wants to reverse. “As Amazon has grown, we’ve seen some of our ICs and SDMs become less connected to their customers and business,” the memo says, referring to individual contributors and their managers. Engineers closer to their products and businesses, it says, “are able to take the most advantage of AI and agentic software development”.

A guideline published as a model

Amazon published the memo saying that sharing its vision for the role “can serve as a model for our customers”. A day later Amazon cut about 250 roles in its devices and services unit, mainly in software development, as Reuters reported. An Amazon spokesperson quoted by Reuters said the cuts would help the company “operate more efficiently and better align with our product roadmap”.

For people directors who have added AI usage to dashboards or performance conversations, Amazon is now a large employer saying in public that those numbers are disconnected from results, and that it will keep using them while it looks for what the memo calls “better metrics” and “smarter goals”. The measure it has added alongside them is an engineer’s ability to explain where AI falls short, and marking that test takes a manager who has used the tools on real work.