GPU servers for AI training

Start with the node, not the cluster

Training scales in two directions: inside a node, where accelerators synchronise at full interconnect speed, and across nodes, where the network takes over. The eight-GPU node is the industry's standard building block because most frameworks are tuned for it; our range's listed example is the Lenovo ThinkSystem SR675 V3, a 3U system with eight GPUs on dual AMD EPYC hosts, positioned by its line for AI training and HPC.

Smaller training programmes — fine-tuning, smaller models, research groups — often land better on 2-GPU or 4-GPU 2U nodes bought in pairs. The deciding figure is the largest model you must fit and train at once; everything else follows from it.

Accelerator memory is the hard constraint

Parameters, gradients and optimiser state must fit in accelerator memory or be sharded across devices. State the model size and the framework in the inquiry: those two facts decide the accelerator class and count more honestly than any benchmark table. Host memory and local NVMe storage then size around the data pipeline — training nodes starve on slow data long before they starve on compute.

Power and cooling decide the chassis

A loaded training node is the heaviest continuous load in the building. The 3U and 4U chassis in our GPU range exist because that load needs power delivery and airflow a 2U box cannot always give. Confirm the per-rack budget at the delivery site before the configuration is priced; the quote's PSU and rail choices depend on it.

What to send with a training inquiry

• The framework and the largest model you will train.
• The accelerator count per node you are planning, or the total compute you are targeting.
• The per-rack power and cooling available at the site.
• Whether the order is a first node or a cluster programme — cluster orders are quoted as programmes with a frozen configuration.

Staging a training purchase

Training programmes de-risk themselves by staging. A first node, configured exactly as the cluster nodes will be, proves the stack — framework versions, data pipeline throughput, real power draw under load — against measurements rather than projections. The cluster order then freezes that proven configuration, so node forty is node one's twin, and the programme's surprises are all in the first chapter where they are cheap.

The staging conversation belongs in the first inquiry: evaluation node now, rollout quantities and phasing stated, configuration frozen between them. Priced as a programme, the two stages share one paperwork chain — one written configuration, serial numbers per unit from the first crate onward.

Keep the paperwork with the programme

Training programmes outlive their first operators. The configuration documents, the serial register and the staging measurements from the first node are the programme's memory; file them where the second and third operators will find them.