14 Beyond the Chatbox: What Long-Running Agents Mean for the Future of Computing

As AI models shift from immediate prompt-and-response supervision to multi-day autonomous agents, current infrastructure encounters major bottlenecks. The tension lies in balancing chip design, memory bandwidth, and tool execution speeds to keep pace with these self-correcting workflows.

In a joint discussion at GTC, Google Chief Scientist Jeff Dean and NVIDIA Chief Scientist Bill Dally address how computing infrastructure must adapt. They explain that traditional computing setups were built for human speeds, but autonomous agents execute operations much faster. Achieving high performance requires co-designing hardware and software by reducing off-chip communication delays, utilizing specialized memory configurations, and applying AI to streamline semiconductor design.

• Inference accounts for roughly ninety percent of the energy consumed in modern data centers.
• Moving a single piece of data from external memory uses a thousand times more energy than the mathematical operation itself.
• Specifying research spaces in natural language enables models to run dozens of experiments autonomously.
• Existing software tools must be re-engineered because slow tool startup times bottleneck fast AI agents.
• Personalized AI tutors and health coaches represent major potential benefits for society by adapting directly to individual human needs.

NVIDIA uses internal tools like Chip NeMo, fine-tuned on proprietary RTL code and design documents, to help junior designers understand complex chip architectures without requiring continuous oversight from senior engineers.

If moving data is the primary energy cost in AI processing, how should future chip architectures prioritize memory placement over raw compute power?