H100 Rental: How to Evaluate Cloud Providers Before You Commit

Picture a two-person ML team that finds an H100 listing on a Friday, pays for a week of capacity, and spends Monday discovering that the GPUs are PCIe cards inside a shared VM, the interconnect cannot carry their multi-node job, and the "available" badge was a cached number from days earlier. Nothing about the offer was dishonest. The team simply never asked the questions that would have exposed the gap between the listing and their workload.

This playbook is for founders, ML leads and infrastructure buyers comparing H100 rental offers for training, fine-tuning or inference. It gives you a scorecard you can run against any provider in an afternoon, plus a short pilot plan to confirm what the paperwork claims. The ranking stays with you, based on your own workload.

1. Decode what the listing actually includes

Two offers can both say "H100" and behave like different products. Start by writing down the hardware facts for each listing in the same format so you can compare them line by line.

• Form factor: SXM or PCIe. SXM modules sit on an HGX baseboard with NVLink between GPUs, which matters for tensor and pipeline parallelism. PCIe cards have different power and bandwidth characteristics.
• GPU memory per card and GPUs per node. A full 8-GPU node is the usual unit for serious training. Fractional nodes can be fine for inference or experiments.
• Inter-node interconnect: InfiniBand, RoCE Ethernet, or plain Ethernet. Ask for the fabric type, link speed per GPU and whether the fabric is shared with other tenants.
• Local storage: NVMe capacity per node, and whether it is local or network-attached.
• Shared storage: whether a parallel or network filesystem is included, and how dataset staging works.
• CPU cores and system RAM per node. Data loaders and preprocessing can starve fast GPUs if the host is undersized.
• Software baseline: OS, driver version, CUDA version, and whether you can change them.

If a listing leaves any of these blank, put a question mark in your scorecard and email the provider. The speed and precision of the answer is itself a data point.

2. Score the access model: bare metal, VM or container

The access model decides how much of the machine you control and how much performance variance you inherit from the platform.

• Bare metal: you get the whole node with root or SSH access. Best for custom drivers, kernel tuning, profiling tools and predictable multi-node performance.
• Virtual machine: easier provisioning and snapshots, with a hypervisor layer between you and the hardware. Ask how GPUs and network adapters are passed through.
• Container or managed job platform: fastest path to running code, least control over the host. Good for inference and short experiments, harder for debugging distributed training.

Match the model to your workload. A team running large multi-node jobs that needs to profile NCCL traffic will usually want dedicated nodes. For a deeper walkthrough of that path, see our guide to evaluating bare-metal GPU cloud providers. Also ask whether the node is single-tenant. "Dedicated GPU" and "dedicated node" are different promises.

3. Find out whether availability is live or confirmed at order

Availability numbers on a pricing page can mean very different things. Some are a live feed from the scheduler. Many are a recorded snapshot that the provider confirms only after you place an order. Both are legitimate, as long as the provider tells you which one you are looking at.

• Ask directly: is the availability shown live inventory, or is it confirmed when the order is accepted?
• Ask what happens between payment and a usable node: provisioning steps, burn-in, network setup and handover.
• Ask whether a paid order guarantees a resource, or whether it starts a confirmation process that could end in a refund.
• Ask how quotes are revalidated if capacity or pricing changes between your first look and your commit.

As an example of an explicit model, the Pacific Compute catalog of dedicated bare-metal GPU nodes lists H100 SXM nodes with 80 GB per GPU and 8 GPUs per node, with InfiniBand interconnect, SSH access, NVIDIA drivers and CUDA installed. Its preview rates are indicative until the allocation is confirmed, availability is recorded and then confirmed when an order is accepted, and a review step revalidates capacity, the current quote and terms before you commit. It also states plainly that a paid order is not yet a ready resource. That level of clarity is what you want from every provider on your list.

4. Compare term options and the cost drivers behind them

Most H100 rental offers come in two shapes: on-demand capacity you can start and stop, and reserved terms where you commit to a period in exchange for better economics or firmer allocation. Some providers offer reserved blocks only by request after a conversation.

• List each term length offered and whether it is self-serve or by request.
• Check minimum commitment: per node, per GPU, per hour or per day.
• Ask what happens at the end of a reserved term: automatic renewal, conversion to on-demand, or shutdown.
• Ask whether you can extend mid-term and whether the same nodes are kept.

When you compare total cost, look past the headline rate. The real drivers are utilization (idle reserved nodes still cost money), egress and storage charges, the time your engineers spend on setup and debugging, the cost of failed or restarted runs, and whether multi-node scaling is efficient enough that more GPUs actually finish the job sooner. Our breakdown of reserved versus on-demand GPU capacity goes further on matching term length to workload shape. For a broader view of how H100 rental compares with larger reservations and what moves pricing, see Pacific's GPU rental guide covering H100 rental, B300 reservations and pricing factors.

5. Run a short benchmark pilot before any long commitment

Specs tell you what a node should do. A pilot tells you what it does with your code. Keep it short, scripted and repeatable so you can run the identical plan on each finalist.

• Day one: verify the hardware. Run nvidia-smi topo to confirm NVLink topology, check driver and CUDA versions, and confirm CPU, RAM and NVMe match the listing.
• Run NCCL all-reduce tests within a node and across two or more nodes to measure real interconnect behavior.
• Run a reduced version of your actual training or inference job and record throughput, step time and GPU utilization.
• Stage a realistic dataset and time the transfer, both into local NVMe and into any shared storage.
• Leave a job running overnight and check for ECC errors, thermal throttling, dropped links or node reboots.
• File one deliberate support ticket and time the first useful response.

Write the results into the same scorecard as the listing facts. If a provider will not allow a pilot shorter than its standard minimum term, weigh that against how much risk you are willing to carry on an unproven setup.

6. Confirm data handling at the end of the rental

Rented nodes will hold your checkpoints, datasets and sometimes credentials. What happens to that data after you leave is a security question, and for many teams a compliance one too.

• Ask whether local NVMe is erased when the resource ends, and by what method.
• Ask whether erasure is documented or attestable if a customer or auditor asks.
• Ask how long data on shared storage persists after the term ends, and who can access it in that window.
• Confirm you have time to copy out checkpoints before the node is reclaimed.
• Rotate any keys or tokens you placed on the node at the end of the rental.

A provider with a clear answer, such as local NVMe erased when the resource ends, saves you a round of security review. A vague answer should push that provider down your list.

7. Test support and failure handling

GPUs fail. NVLink errors appear, InfiniBand ports flap, and nodes drop out of a cluster mid-run. You are buying the provider's response to those events as much as the hardware.

• Ask how hardware faults are detected: provider monitoring, customer reports, or both.
• Ask what the replacement process is for a failed GPU or node, and whether billing pauses while a node is down.
• Ask what support channel you get: ticket queue, shared chat, or a named engineering contact.
• Ask whether support engineers can help with fabric and NCCL issues or only with hardware swaps.
• Ask how planned maintenance is communicated and how much notice you get.

Write the answers down and keep them with your order. If the support commitments are not in the terms, assume they are best effort.

8. Watch for red flags

Any one of these does not disqualify a provider on its own. Two or three together usually mean the offer will cost you more time than it saves.

• The listing says "H100" but cannot tell you SXM or PCIe.
• Interconnect is described only as "high speed" with no fabric type.
• Availability is presented as live but orders frequently bounce or get delayed without explanation.
• Payment is taken with no stated process for what happens if capacity cannot be delivered.
• No written answer on data erasure at the end of the rental.
• The provider refuses any pilot, benchmark or verification access before a long commitment.
• Terms allow the provider to substitute different hardware without notice.
• Support response times are undefined and billing continues during outages.
• Pressure to commit quickly because capacity is "about to disappear," with no documentation to back it up.

If you are comparing H100 rental against larger reserved clusters, bridge capacity while a facility is delayed, or colocating hardware you own, book a 30-minute evaluation call with Pacific Intelligent Technologies, Inc. to walk through your workload, term options and how allocation is confirmed before you commit.

FAQ

Is SXM or PCIe better for H100 rental?

For multi-GPU training, SXM is usually the better fit because the GPUs connect over NVLink on an HGX baseboard, which helps with parallelism across the eight GPUs in a node. PCIe cards can work well for inference, smaller fine-tuning jobs and workloads that fit on one or two GPUs. Decide based on how your job communicates between GPUs, then confirm the form factor in writing.

How long should an H100 benchmark pilot run?

Long enough to verify the hardware, run interconnect tests, run a scaled-down version of your real job and leave something running overnight. For most teams that fits in a few days. The key is running the identical script on each finalist so the comparison is fair.

Does a paid H100 rental order mean my nodes are ready?

Not always. Many providers confirm availability when they accept an order, then provision and hand over nodes afterward. Ask each provider whether its availability is live or confirmed at order, what steps happen between payment and access, and what happens if capacity cannot be delivered.