Power & Thermals for 4-GPU AI Rigs: 240V, PSU Sizing, Undervolting

The 4-GPU Power Reality Check

Running four high-end GPUs for local LLM work is not just about VRAM and tensor parallelism. The electrical and thermal engineering is equally demanding. A single RTX 5090 can draw 575W under sustained load, with transient spikes exceeding 1000W for a few milliseconds. Multiply by four, and you are looking at a peak system draw of 3600-4000W, not counting the CPU, RAM, drives, and fans. Standard 120V 15A circuits in North America deliver only 1440W continuous (80% derating). Even a 120V 20A circuit gives only 1920W. You will trip breakers on startup. The solution is a dedicated 240V circuit. In the US, a 240V 30A circuit provides 5760W continuous, giving ample headroom. For 4x RTX 5090 or 4x H200 builds, this is non-negotiable. In Europe and other 230V regions, a 16A circuit yields 3680W, which is marginal. Upgrade to 32A if possible. Always consult a licensed electrician. The cost is trivial compared to the hardware.

# Quick power budget calculator (example for 4x RTX 5090)
# TDP per GPU: 575W
# CPU + system: 300W
# Total sustained: 4*575 + 300 = 2600W
# Transient headroom: 1.5x = 3900W
# Recommended circuit: 240V 30A = 5760W continuous (80% of 30A)
echo "Minimum PSU rating: 3000W (dual 1500W or single 2000W+1000W)"

PSU Sizing: Beyond Sum-of-TDP

Many builders sum the TDPs and add 20%, ending up with a 3000W PSU for four 575W GPUs. That is a mistake. Modern GPUs exhibit power excursions that can exceed TDP by 50-100% for 1-10 milliseconds. These transients are invisible to typical multimeters but can cause voltage droop and system instability. A quality PSU must have enough bulk capacitance and transient response to handle these spikes. For 4x RTX 5090, a single 2000W PSU is insufficient. The industry standard is dual 2000W 240V PSUs in a 2+2 or 3+1 redundant configuration. For example, Supermicro PWS-2K02A-1R or Seasonic Prime TX-2000 can be paralleled via a PDU. Each PSU runs at 50-60% load, maximizing efficiency (Titanium rating, >94% at 50% load). For 4x H200 (700W TDP each), you need dual 2400W PSUs. Always leave at least 20% headroom above peak transient load. Do not cheap out on PSU brand. Stick to Supermicro, Seasonic, EVGA (SuperNOVA), or Corsair AX series. Use 240V input only, as 120V halves the current rating and increases I^2R losses in cables.

# Example PSU configuration for 4x RTX 5090
psu_1:
  model: Seasonic Prime TX-2000 (240V)
  rail: +12V single rail, 166.6A
  efficiency: 94% at 50% load
psu_2:
  model: Seasonic Prime TX-1600 (240V)
  rail: +12V single rail, 133.3A
  efficiency: 94% at 40% load
load_balancing:
  method: PDU with load-shedding (e.g., CyberPower PDU20SW16AT)
  gpu_power: 4 x 575W = 2300W (peak 3500W)
  cpu_system: 300W
  total_peak: 3800W
  headroom: 20% -> 4560W available from dual PSUs (2000+1600=3600W continuous, 2000+2000=4000W continuous better)

Undervolting: The Free Performance Tweak

Undervolting is the single most effective way to reduce power and temperature without sacrificing performance. NVIDIA GPUs are voltage-binned at the factory, often shipping with higher voltages than necessary to guarantee stability across all chips. By lowering the voltage curve, you can drop power consumption by 25-30% while losing less than 2% in clock speed. On an RTX 5090, undervolting from 1.05V to 0.925V at 2.5 GHz can cut power from 575W to 420W. For a 4-GPU build, that saves 620W total, reducing thermal load and allowing denser packing. The tool of choice is MSI Afterburner (Windows) or nvidia-smi with custom clock and voltage locks (Linux). On Linux, you can use the nvidia-smi command to set a power limit and then manually adjust clock offsets. For AMD GPUs like MI300X, use ROCm's rocm-smi to set power caps and voltage curves. Undervolt aggressively during inference where memory bandwidth is the bottleneck; for training, you may need slightly higher voltage to maintain core clocks during heavy compute kernels. Always stress-test with a real workload (e.g., llama.cpp inference at max context length) for at least 30 minutes per GPU.

# Linux undervolt example for NVIDIA RTX 5090 using nvidia-smi and nvidia-settings
# Set power limit to 400W (down from 575W)
sudo nvidia-smi -i 0 -pl 400
# Lock clock to 2500 MHz at reduced voltage (requires custom V-F curve via nvidia-settings)
# First, enable persistence mode
sudo nvidia-smi -pm 1
# Use nvidia-settings to set GPU graphics clock offset (e.g., -200 MHz) and memory offset
nvidia-settings -a [gpu:0]/GPUGraphicsClockOffset[3]=-200
nvidia-settings -a [gpu:0]/GPUMemoryTransferRateOffset[3]=-500
# Verify with nvidia-smi -q -d POWER

Thermal Design: Air vs. Liquid for 4-GPU

Four GPUs in a single chassis generate enormous heat. Air cooling works if you have a well-ventilated open bench or a server chassis with high-static-pressure fans (e.g., Supermicro 4U with 80mm fans). However, in a standard ATX case, the top GPUs will recirculate hot air, causing thermal throttling. The RTX 5090 FE uses a dual-flow-through design that exhausts hot air out the back, but in a 4-GPU setup, the top card still ingests pre-heated air. Liquid cooling with a custom loop or AIOs (e.g., using NZXT Kraken G12 brackets) is superior. Each GPU can be kept under 60C even under full load, versus 80-85C with air. This directly translates to higher sustained boost clocks. For 4x RTX 5090, you need at least two 480mm radiators (e.g., Hardware Labs Black Ice Nemesis 480GTX) with push-pull fans. The coolant temperature delta over ambient should be under 10C. Use distilled water with a biocide and corrosion inhibitor. For AMD MI300X, liquid cooling is practically mandatory because the TDP is 750W per GPU. Some builders use immersion cooling with dielectric fluid, but that is overkill for most. The key metric is GPU hotspot temperature; keep it below 95C for NVIDIA and below 100C for AMD. Monitor with nvidia-smi or rocm-smi.

# Monitor GPU temperatures and throttling in real-time
watch -n 1 'nvidia-smi -q -d TEMPERATURE,CLOCK,PERFORMANCE | grep -E "GPU Current Temp|GPU Hotspot|Throttle|Power Draw"'

240V Circuit Installation: What You Need

Installing a 240V circuit for an AI workstation is not a DIY job unless you are a qualified electrician. The typical setup is a NEMA 6-30R receptacle (30A, 240V) or a NEMA L6-30R (locking). The breaker should be a double-pole 30A. Wire gauge must be 10 AWG copper for 30A (or 8 AWG for longer runs over 50 feet). The circuit should be dedicated, meaning no other outlets or loads on that breaker. For larger builds (4x H200, 4x MI300X), you may need a 50A or 60A circuit with 6 AWG wire. In Europe, a single-phase 230V 32A circuit with a CEE 7/7 plug is typical. Always install a residual-current device (RCD) for safety. The cost of installation varies from $300 to $1500 depending on distance and local codes. Do not use a 120V to 240V step-up transformer; it is inefficient and can cause ground loops. Instead, have an electrician pull a new circuit from the panel. If you are in a rental, consider a 240V PDU that plugs into a dryer outlet (NEMA 14-30R) but ensure the breaker is not shared. Safety first: any mistake can cause fire or electrocution.

Connector and Cable Management for High Power

The 12VHPWR connector (12+4 pin) used on RTX 4090 and RTX 5090 is rated for 600W per cable, but only if fully seated. Partial insertion causes high resistance, leading to melting. For 4-GPU builds, you need high-quality cables from CableMod or the PSU manufacturer. Avoid daisy-chaining PCIe power cables; each GPU should have its own dedicated cable from the PSU. For 575W GPUs, use two 12VHPWR cables per GPU if possible, splitting the load. Alternatively, use the new 12V-2x6 connector standard which has better sense pins. For AMD GPUs, the 8-pin PCIe connectors are more reliable but still need proper insertion. Use cable combs to keep airflow unobstructed. Measure voltage drop at the GPU with a multimeter during load; if +12V drops below 11.8V, your cables or PSU are inadequate. For 240V PSUs, the input cable must be rated for 15A or 20A (C19 or C21 connectors). Do not use standard C13 cables; they are only rated for 10A. The C19 connector is rated for 16A continuous. Use a PDU with C19 outlets for each PSU.

# Check voltage rails with nvidia-smi (requires root)
sudo nvidia-smi -q -d VOLTAGE
# Example output for RTX 5090:
# Voltage: 1.050 V (current), 0.925 V (undervolt target)
# Also check 12V rail via PSU monitoring (if supported)
sudo ipmitool sensor list | grep -i 12V

Software Power Management: nvidia-smi, Power Caps, and Dynamic Control

Beyond undervolting, software power management allows fine-grained control. Use nvidia-smi to set a persistent power limit per GPU. For example, `nvidia-smi -pl 400` caps the RTX 5090 to 400W. During inference, you can lower it further to 300W with minimal throughput loss because memory bandwidth is the bottleneck. For training, keep it higher. You can also use the NVIDIA Management Library (NVML) via Python to dynamically adjust power limits based on workload. For AMD GPUs, rocm-smi provides similar functionality. In multi-GPU setups, ensure all GPUs have the same power limit to avoid load imbalance. For vLLM or llama.cpp, you can set environment variables like `CUDA_VISIBLE_DEVICES` to isolate GPUs for different models, each with its own power cap. Use `nvidia-smi -pm 1` to enable persistent mode, which keeps the driver loaded and reduces latency. On Linux, you can also use `cpupower` to set the CPU governor to performance and disable C-states to reduce jitter. For ASICs like the Intel Gaudi 3, use the `habana-smi` tool. The goal is to maximize performance per watt, not raw performance. A 400W GPU running at 95% of peak throughput is better than a 575W GPU at 100%.

import pynvml
pynvml.nvmlInit()
handle = pynvml.nvmlDeviceGetHandleByIndex(0)
# Set power limit to 400W (400000 milliwatts)
pynvml.nvmlDeviceSetPowerManagementLimit(handle, 400000)
# Get current power usage
power = pynvml.nvmlDeviceGetPowerUsage(handle)
print(f"Current power draw: {power/1000} W")

Real-World Power Numbers: RTX 5090 vs H200 vs MI300X

Let's compare real power draws. An RTX 5090 at stock pulls 575W under full load (e.g., training a 70B model with tensor parallelism). Undervolted to 0.925V, it draws 420W while losing 1-2% throughput. An H200 (SXM) is rated at 700W TDP, but actual draw during inference with FP8 can be 650W. Undervolting H200 is not officially supported, but you can set power caps via nvidia-smi to 600W with minimal impact. The MI300X is a power hog at 750W TDP; undervolting via rocm-smi can bring it to 600W, but stability varies. For a 4-GPU build, total system power (including CPU, RAM, drives, fans) at the wall: 4x RTX 5090 (stock) = 2300W + 300W = 2600W. With undervolt: 4x420W + 300W = 1980W. That is a 24% reduction. For 4x H200: 4x650W + 400W = 3000W. For 4x MI300X: 4x750W + 500W = 3500W. These numbers determine your circuit and cooling requirements. Always measure at the wall with a power meter. The difference between idle and load is also important; idle power for RTX 5090 is about 30W, H200 idle is 100W (due to HBM3 memory). Use PCIe ASPM and GPU idle states to reduce idle power when not training.

Case Studies: Two 4-GPU Builds Compared

Build A: 4x RTX 5090 in a Fractal Design Meshify 2 XL with air cooling. PSU: dual Seasonic Prime TX-2000. Circuit: 240V 30A. Undervolt: 0.925V at 2.5 GHz. Result: sustained 420W per GPU, GPU temps 78C, hotspot 92C. Fan noise: 55 dBA. Inference throughput on llama.cpp with 70B Q4_K_M: 12 tokens/s per GPU (48 tokens/s total with tensor parallelism). Build B: 4x RTX 5090 in a Supermicro 4U chassis with liquid cooling (dual 480mm radiators). Same PSU and circuit. Undervolt: same. Result: GPU temps 55C, hotspot 65C. Fan noise: 40 dBA. Inference throughput: identical 12 tokens/s per GPU, but sustained for hours without throttling. Build B cost $2000 more but runs cooler and quieter. For training, Build B maintains boost clocks 50 MHz higher, yielding 3% better throughput. The lesson: air cooling works for inference, but liquid cooling is superior for training and longevity. Build C: 4x H200 in a Dell PowerEdge XE9680 with liquid cooling. PSU: dual 2400W. Circuit: 240V 50A. Power cap: 600W per GPU. Result: 48 tokens/s on 70B with FP8 (using vLLM). Temp: 60C. This is the gold standard but costs $200k+.

# Example: Measure sustained throughput with power cap
for gpu in 0 1 2 3; do
  sudo nvidia-smi -i $gpu -pl 400
done
# Run inference benchmark
./build/bin/main -m /models/llama-2-70b-q4_k_m.gguf -n 256 -t 16 -ngl 99 -p "Hello" 2>&1 | tail -n 5

Future Trends: 600W+ GPUs and the Need for 48V Power

The trend is clear: GPU power is increasing. The RTX 5090 is 575W. The H200 is 700W. The B200 (Blackwell) is expected to exceed 1000W. At these levels, 12V distribution becomes inefficient due to I^2R losses in cables and PCB traces. The industry is moving toward 48V power delivery, as seen in the Open Compute Project (OCP) standards. 48V reduces current by 4x for the same power, allowing thinner cables and lower losses. The NVIDIA GB200 NVL72 uses 48V internally. For DIY builders, this means future PSUs will output 48V directly, and GPUs will have onboard DC-DC converters. For now, 240V AC input with 12V PSUs is the standard, but plan your infrastructure to accommodate 48V in the next 2-3 years. Also, expect stricter thermal requirements: liquid cooling may become mandatory for 1000W GPUs. The era of air-cooled 4-GPU builds is ending. Start learning about liquid cooling loop design, quick-disconnect fittings, and coolant maintenance. The myaihardware.com community has guides on custom loops for multi-GPU. Stay ahead of the curve.

Further reading

• NVIDIA GPU Power Management Documentation
• AMD ROCm Power Management (rocm-smi)
• PSU Sizing and Transient Response - Tom's Hardware
• 12VHPWR Connector Issues and Solutions - Gamers Nexus
• Open Compute Project 48V Power Architecture