Your AI Productivity Metric Is Easy to Game
Necati Demir
0:00 / 0:00
Your AI Productivity Metric Is Easy to Game
127 просмотров · 9 дн. назад
Necati Demir
72 подписчика
127 просмотров · 9 дн. назад
“How can I burn more tokens?” Someone asked this recently.
That is the wrong question.
Teams should encourage engineers to learn and experiment with AI. But token usage tells you about activity—not what the activity accomplished.
A rising token count might mean your team is solving harder problems. Or it might mean an agent repeated the same failed attempt 20 times. The dashboard looks successful in both cases.
And once token consumption becomes a performance target, it becomes easy to game.
In this video, I explain why developer productivity cannot be reduced to one metric, why faster code generation can create more verification and review work, and what engineering leaders should examine instead: useful changes reaching users, review burden, rework, reliability, reusable learning, and the total time spent by everyone involved.
This is not an argument against AI—or even against spending more on it. A larger API bill can be reasonable when it produces better results.
But if you measure AI activity instead of progress, you are not leading an AI-first engineering team. You are approving a larger API bill.
Read the article:
https://n.demir.io/articles/burning-m...
Hi, I’m Necati—a computer scientist with 20 years of experience sharing thoughts on tech, AI, and engineering.
TIMESTAMPS
0:00 - Why would anyone want to burn more tokens?
0:16 - Token usage tells us very little
0:34 - DORA and “tokenmaxxing”
0:40 - Token consumption is easy to game
0:50 - Consumption creates the wrong incentive
0:58 - Why one metric cannot measure developer productivity
1:10 - Two developers working on the same bug
1:29 - Inspect the fix, not the token count
1:47 - The score is not the result
1:54 - Faster generation creates more verification work
2:18 - Ten minutes of generation, three hours of review
2:31 - Developers remain responsible for what they ship
2:38 - When a higher AI bill is reasonable
2:50 - The questions engineering leaders should ask
3:09 - Token leaderboards and PR leaderboards
3:19 - What CTOs need to understand
3:30 - AI activity is not progress
#AI #SoftwareEngineering #DeveloperProductivity #AITokens #AICoding #EngineeringLeadership #CTO #TechLeadership #CodingAgents #SoftwareDevelopment #DORA #EngineeringMetrics