AI is making sophisticated analysis easier to access and dramatically cheaper to generate. Stanford’s 2025 AI Index reports that the cost of querying a model at roughly GPT-3.5-level performance on the MMLU benchmark fell more than 280-fold between November 2022 and October 2024.[1] The direction is clear even if the exact benchmark changes: producing another answer, explanation, or set of possibilities keeps getting easier.
An engineer’s attention does not scale the same way. Neither do laboratory capacity, review time, or the number of decisions a team can responsibly make in a day.
That changes the nature of the bottleneck. As AI expands the number of directions we can explore, the scarce resource becomes the deliberate ability to decide which question deserves attention, which evidence is worth gathering, and when a line of reasoning is strong enough to influence real work.
As AI lowers the cost of generating analysis and possibilities, attention becomes more valuable. Engineering teams still have finite time to verify claims, run experiments, and make consequential decisions. The advantage increasingly comes from directing AI toward the right uncertainty and choosing which output deserves real-world follow-through.
In the first article in this series, I argued that we do not need a final verdict on whether AI possesses human-like intelligence before we can evaluate its practical value. For engineering work, intelligence amplification is a more useful frame. The question is whether a person working with the system can reach useful possibilities, evidence, or questions that would have been harder to reach alone.
That capability is becoming inexpensive enough to use almost continuously. A model can summarize unfamiliar material, challenge an assumption, or suggest another way to frame a technical problem in seconds. The cost of asking one more question is often trivial compared with the cost of the engineering work that follows.
This is a real productivity opportunity. In a 2025 Quarterly Journal of Economics study of 5,172 customer-support agents, access to a generative-AI assistant increased issues resolved per hour by 15% on average.[2] The authors are careful about generalizing beyond that setting, and we should be too. Still, the study shows why organizations are excited: under the right conditions, AI can improve a measurable work outcome.
Consider a yield engineer investigating a recurring excursion. The team has an edge-related failure pattern and a few incomplete observations. A general-purpose AI system can quickly produce several plausible explanations. With another prompt, it can expand each explanation into possible checks. One more prompt can generate alternatives to those checks.
Generating another possibility is now cheap. Establishing which possibility deserves action still requires evidence and engineering time.
Every plausible direction creates a burden of evaluation. Someone has to compare the suggestion with the actual process history. Someone has to decide whether the missing evidence can discriminate between competing explanations. If the idea survives that review, the team may need metrology time, a controlled split, or physical analysis before it deserves confidence.
NIST’s Generative AI Profile describes confidently presented false content as a characteristic risk of generative AI systems, including cases where the apparent logic or citations supporting an answer are themselves wrong.[3] In engineering, fluent output is useful for exploration. Evidence is what allows the organization to act.
The underlying problem is not new. In 1971, Herbert Simon described attention as the scarce resource in an information-rich world.[4] More information increases the demand on whatever selects, filters, and interprets it.
Generative AI changes the scale of that problem because information no longer has to arrive from somewhere else. We can manufacture more of it on demand. When an answer feels incomplete, we ask for another. When a hypothesis looks interesting, we ask for five variations. When a report feels thin, we make it longer.
This is where AI can quickly become a noise amplifier. The system is doing exactly what we asked it to do. The failure happens when the organization treats the volume of generated work as evidence of progress. A thousand useful tokens can move an investigation forward. A hundred thousand plausible tokens can leave the team exactly where it started.
In engineering, attention is more than a personal productivity concern. It is a technical resource.
A semiconductor team cannot investigate every plausible mechanism at the same depth. Tool access is finite. Experiments consume lots of engineering hours and schedule. Management attention is finite too, especially when multiple excursions or improvement opportunities compete for the same specialists.
This makes selection part of the engineering method. A good investigation does not expand forever. It becomes narrower as evidence improves. The team identifies the uncertainty that matters most, chooses the comparison that can reduce it, and then updates the direction based on what the system actually does.
AI is extremely useful when it helps enlarge the space early enough to challenge tunnel vision. The actual value appears when the engineer knows when to stop expanding that space and start discriminating among the options.
Research on human-AI collaboration reinforces the need for that distinction. A 2024 Nature Human Behaviour meta-analysis covered 106 experiments and 370 effect sizes. Human-AI combinations performed better than humans alone on average, yet they did not consistently outperform the better of the human or AI working alone. The researchers also found that task type mattered: decision tasks were associated with performance losses relative to the stronger individual baseline, while creation tasks showed a more favorable pattern.[5]
Those experiments span many contexts and should not be read as a verdict on engineering teams or current models. They do make one point difficult to ignore: adding AI to a workflow does not automatically create good human-AI collaboration. Generating material and deciding what deserves action require different disciplines. The first benefits from speed and breadth. The second depends on context, evidence, tradeoffs, and accountability. Engineering organizations need both.
Why this matters? Because organizations tend to measure what they can see. AI adoption is visible. Usage is visible. The number of conversations, prompts, or tokens can be counted immediately. But these metrics say very little about whether engineering performance improved.
If the goal is better engineering work, the measurement has to move closer to the outcome. Did the team reach a supported decision sooner? Did the selected experiment meaningfully reduce uncertainty? Did the resulting action prevent recurrence or protect value? Those are much harder to measure, but they are much closer to why the organization invested in AI in the first place. Otherwise, a team can become extraordinarily efficient at producing activity.
The most valuable AI skill may become the ability to direct attention deliberately.
That starts before the prompt. The engineer needs a useful question, enough context to make the response relevant, and a sense of what evidence would change the current view. During the conversation, suggestions should remain visibly provisional until they connect to observations. At some point, exploration needs a stopping rule, such as this direction now deserves an experiment, this one needs more evidence, and this one is no longer worth the team’s time.
For managers, the same principle applies at a larger scale. The organization needs a way to distinguish an interesting conversation from work that deserves scarce engineering resources. AI can expand the supply of ideas and analysis. Human judgment still governs the commitment of people, equipment, and time.
Deliberate attention means not allowing generation’s availability to determine the work. The engineering question determines the work. AI helps us explore it more deeply.
The first wave of AI adoption has focused heavily on access: who has the tools, who is using them, and how much work they can generate. That was a natural starting point.
The next challenge is harder. We need to become better at deciding what deserves to survive the conversation.
AI gives an engineer more potential routes through an investigation. Some will be familiar, while others will be incorrect. Occasionally, one will reveal the question that transforms the entire direction of the work. The value of that insight depends on whether the team recognizes it, preserves the reasoning behind it, and gives it the attention needed to test it.
As the capacity to generate grows, attention becomes the constraint that determines which possibilities become knowledge and which disappear into the stream of output.
That leads to the next problem in this series: what happens when valuable thinking is discovered inside an AI conversation, and the organization never sees it?
A few operating habits follow directly from this shift:
1. The 2025 AI Index Report – Research and Development. Stanford Institute for Human-Centered Artificial Intelligence. Reports the more than 280-fold decline in cost for a system performing at GPT-3.5-level MMLU performance between November 2022 and October 2024.
2. Generative AI at Work. The Quarterly Journal of Economics, 140(2), 889-942 (2025). Field study of 5,172 customer-support agents; AI assistance increased successfully resolved chats per hour by 15% on average in the studied workflow.
3. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1 (2024; updated publication page 2026). Defines and discusses confabulation risk and the need to manage generative-AI risks in consequential applications.
4. Designing Organizations for an Information-Rich World. Herbert A. Simon (1971). Classic discussion of attention as the scarce resource created by information abundance.
5. When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour 8, 2293-2303 (2024). Meta-analysis of 106 experiments and 370 effect sizes; human-AI synergy varied by task type, with decision tasks showing a less favorable pattern than creation tasks.
