ROCm
ROCm is AMD's open-source GPU computing platform designed as a CUDA alternative for running AI workloads on AMD GPUs.
ROCm, short for Radeon Open Compute, is AMD's open-source software stack for GPU computing. It aims to be a direct competitor to NVIDIA's CUDA platform, enabling developers to run deep learning and LLM workloads on AMD GPUs such as the MI250, MI300X, and Radeon RX series. ROCm includes a HIP (Heterogeneous Interface for Portability) runtime that allows code written for CUDA to be ported with minimal changes, often using automated conversion tools. For AI builders, ROCm supports popular frameworks including PyTorch, TensorFlow, and ONNX Runtime, and it has its own optimized libraries like rocBLAS (BLAS), rocFFT (FFT), and MIOpen (deep learning primitives). In the LLM ecosystem, tools like vLLM and llama.cpp have ROCm builds, though performance and feature parity with CUDA can vary. ROCm is particularly relevant for cost-sensitive deployments where AMD hardware offers better price-to-performance ratios. The platform supports features like FP16, BF16, and INT8 quantization, and on newer GPUs it offers matrix core acceleration analogous to NVIDIA's tensor cores. ROCm's open-source nature allows community contributions and transparency. However, its ecosystem lags behind CUDA in terms of library maturity, debugging tools, and adoption. Installation can be complex on Linux distributions, and Windows support is limited. AI builders should verify that their specific model and inference engine have ROCm compatibility before committing to AMD hardware.
ROCm is not a drop-in replacement for CUDA; many CUDA-optimized kernels require manual tuning or use of HIP ports to run efficiently on AMD hardware.
Related terms
Concepts you'll usually encounter alongside this one.
CUDA is NVIDIA's parallel computing platform and API that allows developers to use GPU accelerators for general-purpose processing, including LLM training and inference.
vLLM is an open-source high-performance LLM inference engine using PagedAttention to efficiently manage KV cache memory and achieve high throughput.
llama.cpp is an open-source C/C++ implementation of LLM inference optimized for CPU and GPU, known for running large models locally on modest hardware with quantized formats.
Mentioned in tutorials
Hands-on guides that put this concept to work.
Stand up a fully functional local LLM stack on Ubuntu 22.04, NVIDIA drivers, CUDA, Ollama, and your first model, in under 30 minutes.
Squeeze a 70-billion-parameter model onto a single RTX 4090 by quantizing weights to 4 bits. Quality stays high, VRAM usage drops to 24 GB.
Scale a single model across multiple GPUs with tensor parallelism. Configure vLLM and llama.cpp, fix PCIe bottlenecks, double your VRAM.
Run multiple isolated AI environments on one host using Proxmox VE with PCIe passthrough. Bare-metal performance, virtual machine convenience.
See how this term affects your memory budget.
Real numbers, real chips, every workload.
Picks for builders, by use case.
Stay Ahead of the AI Curve
Get weekly AI hardware news, benchmark updates, and deals in your inbox. Founding-subscriber list, be one of the first.