AI Training vs. Inference Data Centers: How Site Needs Differ

Training is the process of building a model: thousands to hundreds of thousands of accelerators work on one job for weeks or months, exchanging results constantly. Inference is the process of using a trained model: answering a chatbot prompt, ranking search results, generating code or summarizing a document, usually while a person or application waits for the answer.

Those two jobs put different demands on land. A training run cares about how many accelerators can be wired together in one place, how much power and cooling the site can deliver, and how soon. It does not care much where the end user is. Inference cares about response time and reliability for users in a region, so it behaves more like a cloud or colocation workload than a factory.¹

The balance between them is changing. McKinsey projects that inference will overtake training as the dominant AI workload by 2030, and that inference build-outs are concentrating in metro and near-metro sites with low latency and strong connectivity, while training favors large, high-density campuses.¹ The IEA estimates that data centers used about 415 TWh of electricity in 2024, around 1.5% of global consumption, with the United States accounting for 45% of it.⁸ How that demand splits between training and inference shapes which kinds of land are in demand.

For a broader view of AI campuses, see our guide to AI data center site requirements. This guide focuses on how the two workloads pull site selection in different directions.

Latency is the time a signal takes to travel between two points and back. Light in optical fiber travels at roughly 200,000 km per second, which works out to about 5 microseconds of one-way delay per kilometer of fiber, or about 5 ms for every 1,000 km.⁹ Real routes are longer than straight lines, and every router and switch adds delay on top of that floor.

For a training run, those milliseconds matter inside the cluster, not between the cluster and users. The accelerators synchronize constantly, so the facility is designed for very short, very fast links between racks and buildings. The finished model is then copied to inference sites, and the training campus itself can sit hundreds of miles from the nearest large city.

Inference is the opposite. Each user request makes a round trip, and interactive applications feel slow when network delay stacks on top of model processing time. JLL’s 2026 outlook notes that inference demand requires geographic distribution to reduce latency and serve users, which it expects to drive regional deployments and systems at the edge.⁴ Our guide to latency requirements by workload goes deeper into how many milliseconds different applications can tolerate.

Training rewards concentration. The more accelerators that can be linked in one tightly coupled cluster, the larger the model that can be trained in a given time. That is why training campuses are measured in hundreds of megawatts to gigawatts. The Stargate campus in Abilene, Texas, developed by Crusoe and leased to Oracle for OpenAI workloads, is planned at about 1.2 GW across roughly 4 million square feet.³

Inference rewards distribution. A given volume of inference can be split across many sites, each sized to the demand in its region, and each needing a reliable connection to users and to cloud networks. Individual inference deployments are therefore usually smaller than frontier training clusters, although large cloud regions can add up to very large totals. McKinsey’s analysis expects inference capacity to concentrate in metro and near-metro areas where low latency, connectivity and energy efficiency come together.¹

The practical consequence is that the same acreage can be valued very differently. A 1,000-acre tract next to a high-voltage line in a remote county may suit a training campus well and an inference deployment poorly. A 40-acre parcel near a metro fiber hub may be the reverse. Our guide to what a gigawatt-scale AI campus requires covers the very large end.

On rack density, training and inference have converged. Both increasingly run on the same class of accelerator systems. Supermicro’s GB200 NVL72 rack configuration lists about 132 kW of total rack power with an in-rack coolant distribution unit.⁵ SemiAnalysis puts the NVL72 form factor at about 120 kW per rack, compared with up to 12 kW for a general-purpose CPU rack and about 40 kW for air-cooled H100 racks.²

SemiAnalysis also notes that moving well beyond 40 kW per rack is the main reason liquid cooling is required for these systems, and that many existing facilities cannot support such densities even with direct-to-chip cooling.² For land, that means both workloads now need the mechanical yard space, piping and heat rejection that liquid cooling implies. See air vs. liquid cooling site implications and high-density GPU racks and site power.

Where they differ is how much of that density is concentrated in one place. A training hall may fill entire buildings with these racks on a single fabric. An inference deployment may place a few rows of them inside a metro colocation facility, where the constraint is often the building’s existing power and cooling rather than the land.

Training clusters do not draw power like a typical data center. Because thousands of accelerators compute and then synchronize in step, the facility’s demand can rise and fall together. A PNNL report on large loads states that AI training and inference facilities can inject large active power swings concentrated at specific frequencies over extended periods, which existing grid planning practices do not account for.⁶

Grid operators have responded. In May 2026 NERC issued a Level 3 Essential Action alert on large computational loads, its highest alert level, citing sudden customer-initiated load reductions and significant oscillations, and requiring registered entities to take specific actions.⁷ For a site, this shows up as more detailed utility questions about ride-through, ramp rates and on-site storage or controls, especially for training-heavy campuses. Our guide to NERC reliability and large loads explains the standards side.

Inference load tends to follow user activity through the day, so it looks more like a conventional cloud load. It is still large and continuous, but it is less likely to produce synchronized swings across a whole campus.

Training is becoming more flexible about location, not less. Google’s Gemini technical report states that Gemini Ultra was trained on TPUv4 accelerators across multiple data centers, combining 4,096-chip SuperPods over Google’s intra-cluster and inter-cluster network, and that its network latency and bandwidth were sufficient for synchronous training.¹⁰

Microsoft has gone further. It describes its Fairwater sites in Wisconsin and Atlanta as an “AI superfactory,” connected by a dedicated AI wide area network so that the sites, roughly 700 miles apart, work on large training jobs together.¹¹ The implication for land is that a training campus no longer has to hold an entire cluster on one parcel. It does need very high-capacity fiber to its sibling sites, which is a fiber route diversity and latency question as much as a power question.

Inference has a different kind of flexibility. Requests can be routed to whichever site has capacity, so operators can place less time-sensitive work in cheaper, power-rich locations and keep latency-sensitive work near users. JLL ranks speed to power as the primary site selection criterion, followed by community support, latency and proximity to customers, and notes that grid connection waits in primary markets average more than four years.⁴ That tension is the core of the near-metro vs. remote power-rich sites decision.

Most land can be screened first by asking which workload it fits naturally, then testing whether it can flex to the other. The answer changes who the likely buyer or tenant is, what they will pay for, and which risks they will scrutinize first.

1. 01Size the power: how many MW can the site plausibly receive, and on what schedule? Training needs large blocks; inference can start smaller.
2. 02Measure the network: distance and route to the nearest metro, cloud region and internet exchange, and how many independent fiber paths exist.
3. 03Check cooling: space and water or dry-cooling options for liquid-cooled racks at 100 kW-plus.²
4. 04Ask about load behavior: what the utility will require on ramp rates, ride-through and oscillation controls.⁶⁷
5. 05Decide the story: a site pitched as a training campus is judged on power and speed; one pitched for inference is judged on latency and connectivity.

If you are weighing which story a parcel can support, we can get a site reviewed against both.