RTX Optimization for LLM Inference: Undervolting, Fans, Power Limits

Why optimize at all

Stock NVIDIA cards are tuned for short-burst gaming workloads, clocks at maximum, voltages at safe-for-everyone, fans aggressive. LLM inference is the opposite: sustained, long workloads where you care about steady-state performance, heat, and noise. Out of the box a 4090 will pull 450W transient and 380W sustained on a long generation, mostly because the boost algorithm pushes voltage to keep clocks high. Tune the curve and you can hold 95% of stock performance at 280W and 15°C cooler.

This matters for three reasons. (1) Lower temp = longer card lifetime, especially relevant for used cards. (2) Lower power = lower bill, at 12¢/kWh, 100W saved over an 8h/day load is $35/year per card. (3) Quieter, sustained 380W cooling is loud, sustained 280W is whisper. The work to do this is one-time and takes 20 minutes.

Step 1: Set a power limit (the easy 80% of the gain)

The single highest-leverage knob is the power limit. nvidia-smi lets you cap the card's total board power. Cap it at 70-80% of stock, performance drops by 3-5%, power drops by 25-30%, heat drops dramatically. This is the only tweak that's safe to do on any card with one command.

# See current/max power limit
nvidia-smi -q -d POWER

# Set RTX 4090 to 320W (default 450W)
sudo nvidia-smi -i 0 -pl 320

# RTX 3090 default 350W → 280W
sudo nvidia-smi -i 0 -pl 280

# Verify
nvidia-smi --query-gpu=power.limit --format=csv

# Make it persist across reboot, systemd unit
cat > /etc/systemd/system/nvidia-pl.service <<EOF
[Unit]
Description=Set NVIDIA Power Limit
After=multi-user.target
[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -pl 320
[Install]
WantedBy=multi-user.target
EOF
systemctl enable --now nvidia-pl

Step 2: Undervolt (Linux: lock clock, Windows: voltage curve)

Undervolting goes one step further than power limiting, it lowers the voltage at every clock, so the card stays cooler and pulls less power at the same speed. On Linux without GUI tools this is awkward, nvidia-smi doesn't expose a voltage curve. The workaround is to lock the maximum clock with --lock-gpu-clocks, which has the same effect (lower top clock → less voltage needed). On Windows use MSI Afterburner's curve editor, drag points down by 100mV at high clocks.

# Linux: lock max clock at 90% of boost
# RTX 4090 boost is ~2520 MHz, lock at 2280
sudo nvidia-smi -i 0 --lock-gpu-clocks=210,2280

# Verify
nvidia-smi --query-gpu=clocks.gr,clocks.mem --format=csv

# To reset:
sudo nvidia-smi --reset-gpu-clocks

Step 3: Fan curve

Default NVIDIA fan curves are conservative, they hold the card cool at the cost of being loud. For a server in another room you don't care about noise, so push the curve aggressive (60°C → 60% fan, 75°C → 90%). For a desktop next to you, the opposite, let it run a bit warmer (75°C target) for silence. Either way the goal is steady-state, not the gaming pattern of low-low-low-MAX-MAX.

# Set static fan speed (needs nvidia-settings + X)
nvidia-settings -a [gpu:0]/GPUFanControlState=1
nvidia-settings -a [fan:0]/GPUTargetFanSpeed=70

# For headless servers, use nvidia-fan-control script:
# https://github.com/foundObjects/nvidia-fan-control
# Lets you set per-temp fan curves from a config file.

Step 4: Run the same workload before/after and measure

All this is theoretical until you measure. Pick a representative workload, Ollama running a 30B model for 5 minutes, and capture power, temperature, and tokens/sec before and after. You're looking for: power down 20-30%, temp down 5-10°C, tokens/sec down less than 5%. If tokens/sec drops by 15%+ you went too far on the power limit, back it off.

# Before settings change, baseline
for i in {1..3}; do
  ollama run qwen2.5:32b --verbose "Write 500 words about the GPU memory hierarchy." 2>&1 | tail -5
  sleep 30
done

# Apply your power limit / clock lock, then re-run.
# Compare 'eval rate' lines.

Step 5: Long-term: paste and pads

If you're running a card 24/7, the factory thermal paste degrades in about 18 months. Re-pasting with Thermal Grizzly Kryonaut or PTM7950 drops core temperatures another 5-8°C and is a $15 part. For 3090 FE cards specifically, the original VRAM thermal pads are notoriously thin, replacing them with 2mm Gelid Extreme pads can drop memory junction temperature from 110°C (dangerous) to 80°C (safe). This is the highest-impact maintenance you can do for used cards.

Step 6: Persistence

Your power limit doesn't survive a reboot by default. The systemd unit from Step 1 handles power limit, but clock locks need their own. Either combine everything into a single oneshot service or run it from /etc/rc.local (old school but works). Once persistent, you're done, set and forget.