The new equation:
What AI leaders need to know about infrastructure in 2026

Inference has overtaken training as the dominant AI workload, and for most teams running at scale, it's now the single biggest cost driver. The energy wall is real. And the rules just changed, whether you're ready or not.

Something shifted in 2026. AI projects that once lived in sandboxed experiments are now the core of enterprise products; the brains of customer-facing agents, the engines of competitive differentiation. And as that shift happened, a reckoning arrived: most infrastructure simply wasn’t built for this moment.

That tension, between the ambition of AI-native companies and the reality of what today’s infrastructure can actually deliver, was at the heart of our recent webinar, The State of AI Infrastructure in 2026. We sat down with Erwan Menard, SVP of Product, and Kyle Sosnowski, VP of Cloud Engineering, to dig into the data from our 2026 AI infrastructure trends report and what it means in practice.

What emerged was a frank conversation about a new set of constraints (energy, data center density, silicon supply) and what it actually takes to build AI infrastructure that wins at scale. Here are the six themes that stuck.

1. Inference has overtaken training. Is your stack ready?

For years, the AI infrastructure conversation centered on training: GPU clusters, frontier model runs, the race to scale compute. But the frontier has moved, and the milestone is already behind us.

This isn’t just a technical footnote. If you’re running an AI-powered product at scale, inference is almost certainly your single largest infrastructure cost. Shaving even 10–20% off that number doesn’t just improve margins, it reshapes your competitive position and expands your addressable market.

The challenge: optimizing inference well requires a depth of expertise most teams can’t build in-house. That’s the gap Crusoe Managed Inference is designed to close. Erwan walks through the full picture in the clip below.

2. The secret sauce is actually science (and a bit of art)

Ask Kyle about inference performance and he won’t just cite benchmark numbers. He’ll tell you why those numbers are achievable, and why getting there requires far more than plugging in an off-the-shelf inference engine.

“Spec decoding, intelligent KV cache management, sharing memory across the GPU cluster… it’s an amalgamation of techniques combined with deep expertise in systems and kernel engineering that lets us squeeze out a really low price for high performance.” — Kyle Sosnowski, VP of Cloud Engineering

The result is up to a 9.9x improvement in time to first token compared to standard vLLM. In a world where users are already impatient with AI latency, that gap is felt in chat interfaces, in perception models for autonomous systems, and in every application where responsiveness determines whether users come back.

Kyle’s view of where this is all heading: “The future of compute is memory bound, not CPU bound.” Teams investing now in memory-aware inference architectures will have a structural advantage as models grow larger and agents grow more complex.

3. The energy wall nobody in software planned for

Here’s a question most software engineers have never had to answer: how do you power 120 kilowatts in a single rack?

Five years ago, a standard hyperscaler rack ran at 10.5 kilowatts. Today, an NVIDIA NVL72 GB200 cluster runs at 120 kW per rack. The data center you designed in 2019 cannot support the hardware you need in 2026, and that’s before accounting for grid constraints, curtailment requests, and cooling profiles. Erwan made the point here:

“The actual wall we’re all running towards is an energy wall. As a person who built software products, cloud services, and agents for years, I never thought I would need to care about this.” — Erwan Menard, SVP of Product

But it’s not just a power problem; it’s a synchronization problem. The people responsible for different layers of the stack operate on fundamentally different timescales:

The teams that internalize this constraint now, and choose infrastructure partners who have already solved for it, are the ones who won’t be scrambling for capacity when they need it most.

4. The new standard for infrastructure: “It just needs to work” 

Kyle has a direct way of describing what engineering teams actually want from their infrastructure: they want to be completely ignorant of it.

Not in a naive way. In the way that a developer writing application code shouldn’t have to think about firmware updates, node failures, cooling systems, or grid curtailment. That complexity belongs in the infrastructure layer; invisible, handled, resolved.

That philosophy is what led to Crusoe Command Center: end-to-end telemetry and observability across the full infrastructure stack, from building cooling systems to running workloads. When a node fails, it’s automatically pulled from rotation, replaced from the capacity pool, and you get an email. No war room. No downtime spiral.

In a world where GPU time is expensive and engineering cycles are scarce, “it just works” isn’t a tagline. It’s a competitive advantage.

5. Vertical integration: The architecture of winners

One of the report’s clearest signals: 98% of AI decision-makers rated complete control over their infrastructure as critical for success. That number tells a story about where the industry is heading, and what separates the teams that will scale from those that won’t.

Control doesn’t come from having access to any cloud. It comes from vertical integration; owning and optimizing the full stack from power source to software layer. Erwan made the strategic case simply:

Kyle made the same case from the engineering lens:

“I can go all the way from the building cooling systems through the rack, from the network switches to the hypervisors, the VMs, and the jobs running on them, and tie all of that together into an optimization that better informs how we purpose-build these data centers. That’s an advantage simply not available anywhere else.” — Kyle Sosnowski, VP of Cloud Engineering

For enterprises evaluating infrastructure partners, this is the question that matters most: how far down the stack does your provider’s visibility and control actually go?

6. The question every AI leader should be asking

Every era of cloud computing has had its defining constraint. For the first generation, it was compute. For the second, it was networking. For AI infrastructure in 2026, it’s all three at once, plus energy.

Erwan calls it “the new equation,” and it’s a useful frame for every leader making infrastructure decisions right now:

“If you’re beyond proof-of-concept and hitting real production scale, ask yourself: am I working with people who, one, two, three years down the road, can master this new equation of energy supply, data center supply, and GPUs and cloud services?” — Erwan Menard, SVP of Product

It’s a question worth sitting with. Infrastructure decisions made in 2026 have a multi-year impact. Choosing a general-purpose cloud built for yesterday’s workloads might work today, but it creates meaningful exposure as the constraint environment tightens.

Kyle’s version of the destination lands with characteristic directness:

The AI natives building on purpose-built infrastructure are the ones who will be fastest to production, most efficient at inference, and best positioned to scale as the equation keeps changing. The question is whether you want to get ahead of it or catch up to it.

Watch the full webinar on demand

These clips capture the highlights, but the full conversation goes deeper into speculative decoding benchmarks, the mechanics of Command Center, and what Crusoe’s customers are seeing in practice as they move from POC to production at scale.

Watch the State of AI Infrastructure in 2026 and download the full report to evaluate the trends for yourself.