GPU cloud knowledge base

A100 vs H100 compared on memory, bandwidth, FP8, NVLink and NVIDIA's published speedups, plus when the older Ampere GPU is still the right call.

What the NVIDIA V100 is, its specs, and how it compares with the A100 on memory, bandwidth, Tensor Cores and NVLink, plus whether it is worth using in 2026.

AMD Instinct MI325X specs, how it differs from the MI300X, how it compares with the H200, MLPerf v5.0 status, and when it is the right GPU.

AMD Instinct MI355X specs, MLPerf Inference v6.0 results, MI355X vs MI350X and B200, power and cooling needs, and when it beats an MI300X.

AMD Instinct MI400 series explained: MI455X specs, the MI450 name, Helios rack details, AMD's claims versus Vera Rubin, and dated availability status.

AWS Trainium3 vs NVIDIA B200 and H200 in 2026: published specs, Trn3 UltraServer scale, Neuron software, and who should pick which chip.

Axolotl, Unsloth and torchtune compared on methods, hardware, config style and project status. Note that torchtune is no longer actively maintained.

B300 vs B200 compared on memory, FP4, attention, FP64, power and networking, using NVIDIA's own figures, plus which GPU fits which workload.

Cerebras WSE-3 and CS-4 against NVIDIA H100 and B200: specs, vendor speed claims, what is independently measured, and where Groq fits in.

Every consumer, workstation, unified-memory and edge card for local AI in 2026 in one table: memory, bandwidth, best use, and when renting beats buying.

A map of every datacenter AI chip in 2026: NVIDIA Hopper to Rubin, AMD Instinct, TPU, Trainium, Gaudi 3, Cerebras, Etched and Ascend, with specs and status.

What DeepSpeed does, how ZeRO stages 1, 2 and 3 cut GPU memory, working configs and launch commands, and when FSDP is the better pick.

NVIDIA DGX Spark vs Apple Mac Studio (M5 Max, M5 Ultra) for local AI: memory, bandwidth, price, MLX and CUDA support, and what fits.

DPO vs PPO for aligning LLMs: how each works, what the DPO paper claims, memory cost per method (computed), TRL code, and when to pick which.

Etched Sohu is a transformer-only ASIC claimed to beat NVIDIA H100 at inference. What is claimed, what has shipped, and what is independently verified.

FlashAttention 2 vs 3 compared: what each changes, the published H100 numbers, GPU support, how vLLM and SGLang pick one, and where FlashAttention 4 fits.

What FlashInfer is, how it differs from FlashAttention, its features, install commands, supported GPUs, and how to select it in vLLM and SGLang.

GB300 NVL72 vs GB200 NVL72 compared on memory, FP4, networking and MLPerf results, with what changed, what did not, and which rack fits which workload.

GRPO trains reasoning models with no value network: sample a group, score it, normalize rewards. Worked math, paper settings, TRL code and VRAM.

H100 and H200 come as SXM, NVL and PCIe cards. Specs, NVLink bandwidth, power and which form factor fits inference, training or an air-cooled rack.

H100 vs H200 compared: 80 GB vs 141 GB, 3.35 vs 4.8 TB/s, identical compute, NVIDIA's MLPerf deltas, and which GPU wins for inference, training and fine-tuning.

H200 vs B200 vs GB200 compared on memory, bandwidth, FP8 and FP4, NVLink and MLPerf results, with a clear guide to which GPU fits which workload.

HBM3E vs HBM4: the JEDEC specs, what SK hynix, Micron and Samsung ship, and which GPUs use each, from H200 and B300 to MI355X and Rubin.

HGX vs DGX vs NVL72 explained: a baseboard, a branded system and a 72-GPU rack. Specs, power, cooling and which one fits your workload.

Huawei Ascend 910C, 950PR and 950DT against NVIDIA H100, H200 and H20: reported specs, vendor claims, shipping status and software gaps.

Inference engineering is serving trained models fast, cheaply and reliably. The six levers: engine, batching, quantization, parallelism, hardware, autoscaling.

InfiniBand vs Ethernet for AI clusters: Quantum-X800, Spectrum-X, Ultra Ethernet 1.0, the figures vendors publish, and which fabric fits your job.

Intel Gaudi 3 vs NVIDIA H100 and H200: specs, Ethernet scale-out, Intel's own benchmark claims, software, and where Gaudi stands in 2026.

Jetson Orin Nano Super specs, power modes, which LLMs and vision models run on 8 GB, and how to train in the cloud then deploy to the edge.

Jetson AGX Thor specs, the $3,499 developer kit, T5000 vs T4000, what 128 GB runs on-device, and how to fine-tune in the cloud then deploy.

A practical llama.cpp guide: install, run llama-server on a GPU, offload layers, split a model across GPUs, and quantize GGUF files.

LLaMA-Factory install, CLI and web UI, the methods it supports, and the project's own GPU memory table for 7B to 70B fine-tuning.

What llm-d is, how its router, inference scheduler and prefill/decode disaggregation work on Kubernetes with vLLM, and its official quickstart.

How to fine-tune an LLM: Unsloth, Axolotl, LLaMA-Factory, TRL, PEFT, DeepSpeed and Megatron compared, with GPU memory tables from each project's docs.

Every major LLM inference engine in one table: vLLM, SGLang, TensorRT-LLM, llama.cpp, Ollama, NIM, Dynamo and llm-d, with licenses, formats and how to choose.

How LoRA and QLoRA fine-tuning work, which rank, alpha and target modules to pick, and how much VRAM 8B, 32B and 70B models need (worked math).

What Megatron-LM and Megatron Core are, the parallelism they offer, install and quickstart commands, and when to choose them over DeepSpeed or FSDP.

NVFP4 vs MXFP4 explained: block size, scale format, bits per value, accuracy numbers, and which GPUs run each 4-bit format in 2026.

NVIDIA B200 specs, VRAM, DGX B200 and HGX B200 explained: 180 GB HBM3e, NVLink 5, FP4, power needs, and when B200 beats H200 or B300.

NVIDIA B300 Blackwell Ultra specs: 288 GB HBM3e, 15 PFLOPS dense FP4, 1,400 W, DGX B300 and HGX B300 explained, with vendor benchmarks sourced.

NVIDIA DGX Spark specs, the 2026 price increases, what 128 GB of unified memory runs, and the break-even point against renting a datacenter GPU by the hour.

DGX Station GB300 packs a Blackwell Ultra GPU and 748 GB of coherent memory in a desk-side box. Specs, roughly $100K pricing, and the rent vs buy break-even.

What NVIDIA Dynamo is, how disaggregated prefill and decode, KV-aware routing and the Planner work, and how to run it with vLLM or SGLang.

GB200 NVL72 explained: 72 Blackwell GPUs and 36 Grace CPUs in one liquid-cooled rack. Specs, power, performance claims, and when it beats a B200 server.

What the NVIDIA GH200 Grace Hopper Superchip is, its specs (96 GB HBM3 or 144 GB HBM3e, 480 GB LPDDR5X, NVLink-C2C), and when it beats an H100.

NVIDIA's Groq 3 LPU and LPX rack explained: 500 MB SRAM per chip, 256 LPUs per rack, how it pairs with Vera Rubin for decode, the Groq deal, and status.

What the NVIDIA H20 is, its reported specs against the H100 and H200, and its export-license history with dates, as of October 2026.

NVIDIA H200 specs: 141 GB HBM3e, 4.8 TB/s bandwidth, 900 GB/s NVLink and 700 W. What it changes versus the H100, MLPerf results, and when to pick it.

NVIDIA NIM explained: what the microservice container is, the docker run command, NGC key and licensing rules, and when to use it over vLLM.

Rubin CPX explained: 128 GB GDDR7, 30 PFLOPS NVFP4, built for long-context prefill. Its specs, why it left the GTC 2026 roadmap, and what came since.

NVIDIA Rubin explained: 288 GB HBM4, 50 PFLOPS NVFP4, NVLink 6, current shipping status as of October 2026, and what to run on while you wait.

What NVIDIA Transformer Engine does, how FP8 (E4M3 and E5M2) and its scaling recipes work, which GPUs support them, and a minimal PyTorch example.

Vera Rubin NVL72 (formerly NVL144) explained: 72 Rubin GPUs, 36 Vera CPUs, 20.7 TB HBM4, NVLink 6, cooling, and where it stands in October 2026.

Ollama vs LM Studio compared: interface, install, API, GPU support, headless servers and model formats, with a clear pick for each use case.

How PagedAttention, continuous batching, chunked prefill and prefix caching fit together, with KV cache math you can check and the engines that ship them.

Parameter-efficient fine-tuning compared: LoRA, DoRA, adapters, prefix and prompt tuning, IA3, AdaLoRA, VeRA. What each trains and when to use it.

Qualcomm AI200 and AI250 rack-scale inference accelerators: announced specs, what is unpublished, shipping status, and how they compare with NVIDIA.

RTX 3090 specs, why 24 GB and NVLink still matter, what fits in VRAM, and how it compares with the RTX 4090 for local LLMs and fine-tuning.

RTX 4070 Ti Super specs, what 16 GB of VRAM runs for local LLMs and image models, where it falls short of a 24 GB card, and fine-tuning limits.

RTX 4090 specs, 24 GB VRAM math, which LLMs and image models fit, and when fine-tuning on it works or runs out. Computed fits, vendor-sourced specs.

RTX 5060 Ti for AI: 16 GB or 8 GB GDDR7, 448 GB/s, 759 AI TOPS. What LLMs and image models fit in each VRAM size and which version to buy.

RTX 5070 Ti for AI: 16 GB GDDR7, 8,960 CUDA cores, 1,406 AI TOPS. Which LLMs and image models fit in its VRAM, and how it compares to 5080 and 5060 Ti.

RTX 5080 vs 5090 for AI: 16 GB vs 32 GB VRAM, cores, bandwidth, TOPS and power, with what models fit each card and who should buy which.

RTX 5090 specs for AI work: 32 GB GDDR7, 21,760 CUDA cores, 3,352 AI TOPS, plus which LLMs and image models fit in its VRAM.

RTX 4090 vs 5090 for AI: 32 GB vs 24 GB VRAM, cores, bandwidth, TOPS, power and which LLMs, image and video models fit each card. Who should buy which.

NVIDIA RTX 6000 Ada specs: 48 GB GDDR6 ECC, 960 GB/s, 18,176 CUDA cores, 300 W, and which LLMs fit at FP16, FP8 and INT4 in 48 GB.

NVIDIA RTX A6000 specs: 48 GB GDDR6 ECC, 768 GB/s, 10,752 CUDA cores, NVLink to 96 GB, and which LLMs fit at FP16, FP8 and INT4.

NVIDIA RTX PRO 5000 Blackwell specs: 48 GB or 72 GB GDDR7, 300 W, MIG, and what LLMs fit at FP16, FP8 and INT4, compared with the PRO 6000.

NVIDIA RTX PRO 6000 Blackwell specs: 96 GB GDDR7, three editions, MIG, and which LLMs fit at FP16, FP8 and INT4, plus how it differs from the H100.

RTX PRO 6000 Blackwell vs RTX 5090 for AI: 96 GB vs 32 GB, power, ECC, MIG and NVLink, with computed model fit tables from NVIDIA's own specs.

NVIDIA Rubin vs Blackwell vs Hopper compared on memory, bandwidth, NVLink and low-precision compute, with each generation's status and what to run today.

AMD RX 7900 XTX specs for AI: 24 GB of VRAM, what fits, ROCm, llama.cpp and Ollama support on Linux and Windows, and the gaps against CUDA.

AMD Radeon RX 9070 XT specs, what 16 GB of VRAM fits, what runs on ROCm, llama.cpp and Ollama today, and where it trails CUDA.

What SGLang is, how RadixAttention reuses the KV cache, the exact install and serve commands, and when to pick it over vLLM on a rented GPU.

What TensorRT-LLM is, how to install it with pip or the NGC container, serve a model with trtllm-serve, and which NVIDIA GPUs it supports.

What Tenstorrent builds: Blackhole cards, Galaxy and QuietBox, with listed prices and specs, how they compare with NVIDIA H100 and B200, and who they suit.

What NVIDIA Triton Inference Server is, how its model repository and dynamic batching work, a Docker quickstart, and where it fits for LLM serving.

Hugging Face TRL explained: SFTTrainer, DPOTrainer and GRPOTrainer with the docs' own code, dataset formats, LoRA, vLLM and when to use it.

What TTFT, ITL, TPOT and tokens per second mean, how engines define them, the roofline math behind each, and how to measure them with vLLM.

UALink vs NVLink compared: the 200G 1.0 and 2.0 specs, the 1,024-accelerator limit, NVLink 5 and 6 bandwidth, and what AMD Helios changes.

What Unsloth is, how to install it, what VRAM the project says QLoRA and LoRA need, and when to pick it over Axolotl, TRL or LLaMA-Factory.

Install vLLM with uv or pip, run the official Docker image, and set tensor parallelism and quantization, with commands copied from the vLLM docs.

vLLM vs Ollama compared on concurrency, throughput design, quantization, multi-GPU, install and APIs, with a clear rule for when to switch.

NVLink is NVIDIA's GPU-to-GPU interconnect. Bandwidth by generation, how NVSwitch builds an NVL72 domain, and how it differs from PCIe and InfiniBand.

What Ollama is, how it runs models locally, which GPUs it supports, the commands to install it, and when to move it to a rented cloud GPU.

AWS Activate gives startups from $1,000 up to $200,000 in credits. See the tiers, who qualifies, how to apply, what the credits cover and when they expire.

Claude for Startups gives approved founders $1,000 in API credits, a free year of Claude Team and partner offers. Eligibility, how to apply, and the terms.

Google Cloud credits for startups: $2,000 at the Start tier, up to $350,000 for AI-first Seed to Series A teams. Tiers, eligibility, how to apply.

Microsoft for Startups offers Azure credits of up to $150,000. See the credit levels, eligibility, how to apply, what the credits cover and when they expire.

AWS, Google, Microsoft, NVIDIA and Anthropic all run startup credit programs. What each one gives, who qualifies, how long credits last, and which to apply for.

Training teaches a model patterns from data; inference applies what it learned to new data. The real differences in compute, timeframe, and cost.

AMD vs NVIDIA GPUs for AI: architecture, CUDA vs ROCm, memory, pricing, and when each one is actually the right call for training and inference.

The 12 best GPUs for AI and machine learning in 2026, ranked from B200 and H200 down to RTX 4060 Ti, with specs and where to rent each one.

Shared GPU memory is system RAM borrowed over PCIe. Dedicated memory is VRAM on the card. The bandwidth gap decides if your model runs fast or crawls.

A practical comparison of GPU monitoring tools: nvidia-smi, gpustat, nvtop, nvitop and jupyterlab-nvdashboard, with install commands and when to use each.

A quick GPU stress test for Linux using gpu-burn and stress, with htop, iotop, sensors and nvidia-smi watching temperatures and utilization while it runs.

A tokenizer turns raw text into numeric IDs a model can compute on. How word, character and subword tokenization work, and how to train your own.

NVIDIA competitors in AI inference: AMD, Google TPU, AWS Trainium/Inferentia, Intel Gaudi, and startups like Groq and Cerebras, and what each is good at.

ROCm vs CUDA in 2026: performance, hardware support, compatibility, and migration. CUDA leads on ecosystem depth; ROCm has closed the gap on inference.

A practical guide to serving LLMs with vLLM: neural networks and transformers, attention and the KV cache, and hands-on deployment on a rented GPU.

How vLLM, TensorRT-LLM and SGLang differ in architecture, cold start and workload fit, with a decision table and docker commands to run each on a rented GPU.

What GPU nodes, interconnects, storage and orchestration software actually do, and how to pick the right shape of cluster for your workload.

Tokens are the units AI models read, bill and think in. How tokenization splits text, varies by language, and what token limits mean for context windows.

AI inference is what happens after training: a deployed model runs on new data to produce real predictions. How the inference pipeline works.

Fine-tuning adapts a pre-trained model to your data instead of training from scratch. The techniques, the seven-stage pipeline, and when it's worth it.

Jupyter Notebook runs code cell by cell alongside output, docs and visuals. How it fits ML workflows, where it breaks down, and running it on a cloud GPU.

faster-whisper, insanely-fast-whisper and WhisperX all run the same Whisper weights. Here is how their engines differ and which one to pick.

DigitalOcean GPU Droplet pricing per GPU-hour, checked against live rates from nine providers, plus the 5 TiB scratch disk that does not survive a destroy.

The best GPU for LLM inference depends on your segment. H100 for production serving, A100 for training value, L40S for local dev, with real specs and pricing.

On-demand H100, A100, L40S, and RTX 4090 rates from 8 providers' own pricing pages, plus the hidden costs that decide your real bill. Dated August 2026.

H100 rental rates bottomed near $2.79/GPU-hr in mid-2025 and climbed past $2.82 by April 2026. The supply data behind the reversal.

Every real free GPU option for 2026: always-free tiers like Colab and Kaggle, plus application-based programs worth $2K to $350K, with catches named.

Google Colab alternatives compared on pricing, session limits and GPU availability, with exact Colab Pro and Pro+ costs and where a dedicated GPU cloud wins.

The actual arithmetic for LLM VRAM sizing (weights, KV cache, and overhead), plus a model x quantization x GPU table so you stop guessing at capacity.

Complete guide to NVIDIA Inception for AI startups (2026): benefits, eligibility, AWS credit tiers, and how to apply, verified against NVIDIA's own FAQ.

Serverless GPU pricing beats a dedicated box below roughly 69% utilization, above it dedicated wins. Real provider prices and the worked break-even math.

TPU vs GPU compared on architecture, current-gen specs, software, and price: Google's Ironwood TPU against NVIDIA's H100/H200, with a clear decision framework.

What AI cloud and neoclouds actually are, how AI cloud computing differs from general-purpose cloud, key features, use cases, and how to choose a provider.

How to install ComfyUI Manager, fix the missing custom nodes in a workflow, pin a working set with snapshots, and get past the errors that block installs.

Scripting a ComfyUI node and model install with comfy-cli: exact commands, the flag that catches dependency conflicts, and what still won't survive.

The 8 ComfyUI custom node packs almost everyone installs: exact registry IDs, real dependency traps from each repo's own issues, and what breaks.

Running ComfyUI on AMD GPUs with ROCm: which PyTorch build to use, which custom nodes break, and what an MI300X actually does on image and video workflows.

There's no single MI300X-vs-H100 multiplier. Real vLLM, MLPerf, and SemiAnalysis numbers, plus the software gaps that decide the outcome.

Lambda Cloud has no stop button, only launch, restart, or terminate. What that means for your disk, and what a persistent filesystem buys you.

A powered-off Paperspace machine still bills storage, IP and add-ons hourly. What actually stops the meter, and what block storage costs per GB.

Vast.ai bills disk while an instance is stopped, and a Volume can't leave its host. What actually survives destroy, and what doesn't.

What a GPU rental marketplace actually is, the four structural types, and the six things to check before you rent, with live numbers from 744 offers.

Provider snapshots are region-locked and volume-bound by design. What a portable GPU snapshot must capture, and the driver trap that breaks the restore.

Zero percent GPU utilization is also what thinking looks like. How a correct automatic idle stop decides, and why it must refuse to guess.

We pulled every offer from our marketplace feed across 8 providers. Picking the right provider saves less than you think; the right host saves more.

The MI300X has 192GB of HBM3, more than an H100 or H200, from $1.71 to $12/hr. Verified prices, the sparsity trap, and an honest read on ROCm.

RunPod has three storage types and they are not interchangeable. What container disk, volume disk and network volume each keep when a pod stops or terminates.

ComfyUI custom nodes reinstall every session on a fresh cloud GPU. Why it happens, what ComfyUI-Manager's snapshot really saves, and what actually fixes it.

Five real ways to stop paying for a cloud GPU when idle, each mechanism, and the specific catch that keeps your meter running.

Terminate a cloud GPU box and the disk is deleted, with no backup to restore from. Why it happens, why a volume is not a backup, and how to stop losing work.

Moving a GPU workload to another cloud provider is easy to start and hard to finish. What actually has to come with the box, and how to move it clean.

Idle time is 30 to 95 percent of most cloud GPU bills. Here is the actual per-hour and per-month math by GPU class, and where the waste hides.

A checkpoint saves your model, not the machine that trained it. The difference between saving a model and saving the whole training environment, mid-run.

When your provider runs out of capacity and you rent elsewhere, a fresh box makes you re-resolve torch, CUDA and every wheel. requirements.txt won't save you.

A GPU volume is region-locked and bills while idle, which is what keeps you stuck on one provider. Why storage cannot follow you to a cheaper one.

A volume snapshot brings back your files, not your working box: venv, custom nodes, models, OS state. What actually restores your setup.

A reclaimed spot GPU wipes the whole environment, not just data. Why checkpoint-and-restart falls short, and why recovery must be set up before the reclaim.

A preempted spot GPU can vanish with seconds of notice. Why volumes and disk snapshots only bring back data, and what has to capture your box beforehand.

Stopping a pod does not stop the bill, and suspend expires. What idle GPU time really costs, and how to pause a box and resume the exact setup anywhere.

Spot GPUs are cheap until a reclaim wipes your run. Why checkpoint scripts alone fail, and what has to already be capturing your box to survive an interruption.

Committing to one GPU provider means rebuilding everything the day you leave. How to build an environment not locked to one provider's region.

A RunPod network volume bills while idle and cannot leave its datacenter. Six ways to migrate off one, ranked by cost and portability.

How to run ComfyUI online without rebuilding it each time: the same nodes, models and workflows on any cloud GPU you rent, across providers.

Where the ComfyUI models folder lives, why cloud GPUs lose it on every restart, and how to keep checkpoints and LoRAs off the download path for good.

Running ComfyUI in the cloud means rebuilding custom nodes and re-downloading models on every new GPU. How to keep a ComfyUI cloud setup that survives the swap.

Six RunComfy alternatives for running ComfyUI on cloud GPUs, priced and compared: managed clouds, raw GPU pods, buying your own card, and portable studios.

A real training run moved across three GPU boxes: process data on an A100, snapshot, resume training on a 5090, then pick it up again on another VM.

A step-by-step walkthrough of running FLUX diffusion inference faster on a multi-GPU cloud VM with xfuser (xDiT): CUDA setup, install, and parallel inference.

Vertex AI, SageMaker, Azure Foundry, Hugging Face and the GPU-native clouds compared on supply, governance and cost, and which one fits your workload.

Idle compute, not hardware, drives your GPU bill. Make training interruptible, checkpoint properly, and migrate when prices shift: the levers behind a 40% cut.

Shadeform shows you which GPUs are available; Aquanode runs and moves the workload. Where the two differ on deployment, monitoring and workload mobility.

Real hourly rates for H100, A100, 4090 and more on Aquanode vs RunPod, plus the throughput and cold-start costs a price-per-hour table always hides.