A GPU server room starts with engineering limits

A single GPU rack cannot be planned like an ordinary server rack filled with more expensive servers. It has a different power density, sharp load changes, a large stream of hot air, and a network that affects useful performance almost as much as the accelerators. The choice between an integrator's ready facility and an upgrade of your own server room should therefore follow measurable engineering limits, not the location that happens to have free rack units.

My working position is simple: upgrade your own facility when the organization already controls the room's power, cooling, and operations and can prove that all three have enough headroom for the full rack. If each item needs a separate construction project, a ready facility is usually faster and more predictable. Owning the room does not make it suitable for AI infrastructure.

Define the rack profile first

The decision starts with a load specification covering at least the next three years. A list of servers for the first purchase is not enough because changing power lines, pipes, cable routes, and floor capacity after launch costs more than allowing calculated headroom from the start.

Build the profile for every device: rack units, depth, weight, number of power feeds, rated and maximum power, airflow direction, cooling type, network ports, and permitted cable length. Add the switches, storage system, management nodes, console, and CDU separately if liquid cooling is used. They occupy space and release heat even though early budgets often omit them.

The following fragment can go to the equipment supplier, facility engineer, and operations team. The values are examples and must be replaced with figures for the selected configuration:

{
  "rack_id": "AI-R01",
  "design_horizon_years": 3,
  "it_load_kw": {
    "day_one_expected": 24,
    "day_one_peak": 31,
    "final_peak": 46
  },
  "cooling": {
    "mode": "air_or_direct_liquid",
    "heat_rejection_kw_final": 46,
    "required_inlet_c": "per_equipment_spec"
  },
  "power": {
    "feeds": 2,
    "voltage": "per-equipment-spec",
    "redundancy_target": "defined-by-business"
  },
  "network": {
    "fabric_ports": 16,
    "management_ports": 12,
    "storage_ports": 8
  },
  "rack": {
    "usable_u": 42,
    "max_static_load_kg": "verify",
    "max_depth_mm": "verify"
  }
}

Three different power figures matter here. day_one_expected helps estimate normal operation and electricity bills. day_one_peak determines launch readiness. final_peak sets the limit for cabling, power distribution, and cooling. Do not mix them. A facility that accepts an average of 24 kW but trips a feed at 31 kW is not ready for this rack.

Record the operating mode as well. Training can keep accelerators near full load for hours, inference creates a different profile, and a development lab sits idle between experiments. A power cap through the BMC or scheduler is acceptable as a managed policy, but it does not excuse an undersized feed. If that policy fails, the equipment still has to remain in a safe electrical and thermal state.

Size the electrical feed for a failure

The power system is suitable only if it carries the design load after one planned failure. The note "two 32 kW feeds" says nothing until you know whether the sources are independent, how heavily the UPS units are loaded, which breakers sit along the path, and whether the remaining branch can take the entire permitted load.

Start with a single-line diagram from utility power to the rack PDUs. It should show breaker ratings, cable sizes and lengths, UPS characteristics, redundancy groups, bypasses, the generator, and common failure points. For every branch, state the continuous permissible load under local rules and design conditions. You cannot declare the entire breaker rating available as IT power.

GPU systems have several power supplies and several cords. That creates redundancy only with the right distribution. If all cords for one half of a system reach a shared PDU or an upstream breaker, losing the branch may reduce performance or stop the node. A map of "PSU - outlet - PDU - breaker - UPS - feed" is more useful than colored cables because it shows actual independence.

The NVIDIA DGX H100 planning guide gives a maximum of 10.2 kW for one 8U node, while four such nodes reach 40.8 kW per rack before some supporting loads are added. This is not a universal configuration or a recommendation to copy it. It is a useful scale check: a free rack in a corporate server room was often designed for much lower density.

Transients also matter. After power returns, servers, fans, pumps, and network equipment may start almost together. Check staged startup, UPS behavior with a nonlinear load, runtime until the generator is available, and system behavior when utility power returns. Ten minutes of UPS runtime does not help if the chiller or pump loop is outside the coordinated backup scheme.

At a ready facility, ask for assigned power for each rack feed, the redundancy design, permitted connectors, PDU-level metering, and the process for raising the limit. Your own server room needs a signed electrical calculation and a real load test. In both cases, the answer should be expressed in kilowatts and failure scenarios.

Air cooling reaches an airflow limit

An air system must deliver enough cold air to the server inlets and remove the entire hot stream without recirculation. Total cooling capacity for the room does not prove that one dense rack will receive its share.

Almost all electrical power consumed by IT equipment becomes heat inside the room, so an initial calculation treats a 40 kW load as roughly 40 kW of heat to remove. The engineer then adds actual losses and system constraints. Converting the result to BTU/h does not change the physics, only the units: 1 kW of heat is about 3,412 BTU/h.

The NVIDIA DGX H100 facility guide gives 157 cubic feet per minute per kilowatt as a minimum airflow reference, while explicitly requiring adjustment for local conditions, altitude, and the equipment's temperature rise. For a 40 kW rack, that reference already exceeds 6,000 CFM. It is useful for checking scale, not as a finished technical specification.

ASHRAE Thermal Guidelines separates recommended and allowable inlet temperature and humidity ranges. Do not use the allowable edge as the normal setpoint because it leaves less margin for a clogged filter, fan failure, or hot weather. Measure inlet temperature at the bottom, middle, and top of the rack, not on a wall beside the air conditioner.

Blanking unused rack units, separating cold and hot aisles, sealing cable openings, and setting the correct fan direction prevent recirculation. A side-flow switch in a rack built for front-to-back flow needs ducting or another deliberate treatment. Otherwise, a small network device can feed hot air into the inlet of the adjacent GPU node.

It is dangerous to judge a comfort air conditioner by the number on its nameplate. It may perform poorly around the clock, fail to maintain the required airflow, lack a winter kit, remain off after a brief power loss, or fail to report an alarm to monitoring. In your own server room, check unit redundancy, automatic restart, condensate drainage, filtration, and maintenance access that does not require shutting down the rack.

A ready facility has an advantage if it already sells confirmed density at the exact position and can show an airflow map. The term "data center" guarantees nothing, since some halls were designed for 5 to 10 kW per rack. Ask for measurements from a nearby comparable load and for the conditions under which the operator will require a power reduction.

A liquid loop changes the responsibility boundary

Liquid cooling makes sense when airflow, noise, or power density prevents the planned configuration from fitting. It does not eliminate air conditioning because power supplies, memory, drives, and network equipment may still release some heat to the air.

Distinguish a cooled rear door from direct liquid supply to components. A rear door extracts heat from exhaust air and usually changes the server less. Direct-to-chip cooling sends fluid to cold plates on processors and accelerators, but it requires agreement on the server loop, CDU, fluid quality, pressure, flow, and temperatures. Immersion changes the equipment design and maintenance process even further. These options do not belong in one line called "water cooling."

ASHRAE material on liquid cooling separates the facility water system from the technology cooling system. A CDU often sits between them, moving heat through a heat exchanger, controlling flow, and isolating server requirements from building water. This division is useful: the facilities team owns the available temperature and heat rejection capacity, while IT operations owns the secondary loop parameters and node compatibility.

Set four boundaries before ordering equipment:

• who supplies and maintains the CDU, pumps, filters, and sensors;
• who is responsible for fluid quality and sample analysis;
• where isolation valves, leak sensors, and emergency drainage are located;
• what happens to compute jobs when a pump or external cooling is lost.

Choose the supply temperature against equipment requirements and dew point. Fluid that is too cold increases condensation risk, especially with humid air and exposed pipework. Insulation, humidity control, and automation must operate as one system. A leak sensor signal should do more than enter a log. It needs a predefined response such as notification, pump shutdown, branch isolation, or controlled job termination.

At a ready facility, confirm quick-connect types, permitted fluids, temperature range, pressure, CDU redundancy, and the number of kilowatts the loop can actually absorb. In your own server room, check whether pipes can be routed safely, whether the building can carry the added equipment, where the heat will go, and who will respond to an overnight incident. If different contractors answer those questions, make the responsibility matrix part of the contract.

The network determines how many GPUs do useful work

Choose the network design for the compute pattern, not for the number of free ports. A single multi-GPU server, a distributed training cluster, and a farm of independent jobs have different latency, bandwidth, and loss requirements.

During distributed training, gradient exchange can quickly make the inter-node fabric the constraint. Adapter and switch speeds must match, capacity must avoid hidden oversubscription, latency must be predictable, and the selected transport must have a tested configuration. A technology name does not prove the result. Run a representative collective operation and compare scaling across two, four, and more nodes.

Separate the management, storage, and compute fabric planes in the design at minimum. They do not always need physical separation, but addressing, access, monitoring, and failure behavior must be clear. Do not leave the BMC on the same reachable network as user workstations. Management ports are still required when compute systems occupy every rack unit.

The external connection affects dataset loading, image updates, backup, and remote work. A large internet circuit will not repair slow storage beside the rack. Measure the full path: data source, array, storage server, switches, network adapters, and file system. Average sequential read speed does not reveal behavior with thousands of small files.

Cable routes need space, bend radius, labels, and acceptable length. In a dense rack, a thick bundle can obstruct hot airflow and access to power supplies. High-speed copper cables have length limits, while fiber adds requirements for modules and clean connectors. Moving the rack to the next room can change the network specification and budget.

A ready facility is easier if the fabric, cross-connects, and external circuits already exist in the required topology. Check which components are dedicated to you, where shared infrastructure begins, and how long switch replacement takes. An upgrade to your own server room gains an advantage from a short path to existing storage only if that storage really sustains the AI workload. Proximity without measurement proves nothing.

The room must support operations

Physical readiness means the rack can be delivered, installed, serviced, and disconnected safely. Access control and a camera do not compensate for a weak floor, narrow door, or lack of space to remove a heavy server.

Check the entire route from the unloading area: opening height and width, lift capacity, thresholds, turns, temporary storage, and space for a server lift. The rack weight must include servers, PDUs, cables, doors, the CDU, and reserve. A building engineer checks distributed and point loads on the floor, not just total weight.

There must be room in front and behind for rail installation, node replacement, and a service cart. A server close to one meter deep may not fit an old cabinet once cable managers are added. Rack doors, a cooled rear door, and floor panels must open without blocking an escape route.

The access system should distinguish entry into the room from work on a particular rack. Record who may enter alone, who needs an escort, how contractors receive temporary access, and how permissions are revoked. Cameras should cover approaches and service work under the organization's retention rules. At a ready facility, ask for visit logs and the process for releasing storage media. At your own facility, tie the procedure to internal security.

Fire protection needs a design compatible with local rules, the equipment, and people in the room. You cannot simply place a gas cylinder beside the rack. Detection, warning, a safe delay, ventilation interlocks, and a clear reentry procedure are required. Protect emergency power-off controls from accidental activation and test them regularly.

Noise is also a safety issue. Dense GPU systems at full fan speed may require hearing protection and limits on time spent nearby. NVIDIA publishes a high sound power level for the DGX H100 and separately calls for personal protective equipment. Measure noise under load and include the result in operating instructions even if you choose different equipment.

A ready facility is usually better prepared for heavy deliveries, scheduled maintenance, and round-the-clock incidents. Your own server room gives the team direct control and fast access, but only with on-call coverage, spare parts, and authority over the engineering systems. An access card does not reduce power recovery time when no electrician is available at night.

Put the integrator and owner boundary in writing

A ready facility removes some construction and operations work, but it does not transfer responsibility for the entire AI system to the operator. The owner remains responsible for data, the software environment, access rights, job scheduling, backups, and performance validation.

Create a RACI matrix for power, cooling, the rack, cables, networking, firmware, spare parts, monitoring, and response. Every incident needs one party that restores service and one decision owner. "Shared responsibility" without named actions usually means delay during a failure.

In a ready facility contract, measurable boundaries matter more than advertised uptime percentages. Where does the power SLA end? Which sensor defines a temperature breach? Who replaces a PDU? How long does admission for your engineer take? Can the operator cap rack power during a cooling problem? Which tasks require a maintenance window?

For your own server room, the same questions move into internal procedures. Building operations and IT often have different priorities. Facilities wants a stable mode and infrequent changes, while IT needs to replace nodes and run tests. Agree on windows, shutdown procedure, spare consumables, escalation contacts, and the authority to stop unsafe work.

Compare both options at the same boundary. If the price of a ready facility includes two feeds, cooling, physical security, and an on-duty team, do not compare it only with the invoice for a new air conditioner in your own room. Add design work, construction, UPS systems, generation, service contracts, monitoring, inspections, spare parts, and employee time.

GSE.kz can take part in such a project as an integrator for AI and data center infrastructure, including server selection and ongoing support. That does not replace engineering acceptance: the customer and integrator should approve the load profile, responsibility boundaries, and test criteria before delivery.

A constraint table makes the decision visible

The facility choice becomes clearer when every requirement has a number, an owner, and evidence. Ratings such as "good" or "bad" hide different risks, so use "confirmed," "work required," and "unknown."

Reduce the comparison to five groups:

• For power, a ready facility should assign a limit to each feed, while your own server room should pass review of the single-line diagram and UPS headroom.
• For heat, the operator confirms the limit at the exact position and its failure mode. The owner of a private room calculates an air or liquid loop and then tests it.
• The ready-facility schedule depends on free space, cross-connects, and a contract. The private-facility schedule depends on design, purchasing, permits, and construction.
• A ready hall offers available circuits but may add cost and latency to the data. A private room shortens the path if storage and the fabric sustain the load.
• Access to an external site follows the operator's procedure. A team enters its own server room faster, but it needs permanent authorization, on-call specialists, and spare parts.

Assess expansion separately. At a ready facility, adjacent racks help only if the operator has reserved power, cooling, and network ports for them. In your own server room, the limits come from the room, electrical feed, and engineering systems of the whole building. A contract divides responsibility at an external site, while internal teams and contractors retain it in your building. Both options need named owners.

A ready facility is usually stronger when launch must happen quickly, the rack exceeds the density of the existing hall, or the building lacks continuous engineering coverage. It also makes sense as a first phase when the actual load profile is not yet measured and the organization does not want to build from assumptions.

An upgrade to your own server room wins when the room already has sufficient feeds and reserve, large datasets sit nearby, policy prevents equipment from leaving, or the team must handle hardware often. The case becomes stronger if the organization plans several racks because capital work is spread over more useful capacity. Those future racks need an approved plan, not a story used to justify extra spending today.

Do not judge the options by one annual total. Separate one-time investment, recurring fees, internal labor, and the cost of launch delay. Then test sensitivity to three events: load growth, schedule movement, and failure of the primary cooling system. The option with the lowest base price may become expensive after the first event.

Acceptance must heat the rack

Documents confirm the intent, while a load test confirms the assembled system. Do not accept a GPU rack merely because its servers turn on. At idle, it consumes far less power and heats the room much less than it does during a sustained compute job.

Run the test in an agreed sequence:

1. Match labels on every feed, PDU, outlet, network port, sensor, and isolation valve against the as-built diagram.
2. Raise load in steps, holding each level until temperature, power, and air or fluid flow stabilize.
3. At design load, record inlet temperatures by rack height, branch power, PSU state, GPU clocks, network errors, and storage speed.
4. Simulate agreed failures one at a time: loss of a feed, cooling module, pump, network path, and monitoring channel.
5. After recovery, check automatic startup, absence of overload, job integrity, and delivery of every alert.

Set acceptance criteria before the test. "Temperature is normal" is not a criterion. State maximum readings at the selected equipment's inlets, permitted imbalance between feeds, absence of thermal clock reduction, minimum throughput, alert delivery time, and job behavior during a failure.

The thermal test must run long enough for the room, water, and building structure to reach steady state. A short run checks only electrical startup. A summer repeat test or review of the calculation may be needed if initial acceptance happened in cold weather and the system uses outdoor air or dry coolers.

Keep the initial measurements as a baseline: firmware versions, BIOS and BMC configurations, power limits, temperature and humidity, PDU charts, GPU telemetry, network counters, and storage test results. After a firmware change, node addition, or airflow modification, repeat the relevant part of acceptance. Without a baseline, the team debates impressions instead of comparing readings.

The facility decision is ready when both options use the same specification and one of them has proved that it passes failure and thermal scenarios. If a supplier or internal team cannot state the available kilowatts, removable heat, and owner of incident response, that is already an assessment result. The facility is not ready for a GPU rack, regardless of how much free space its plan shows.