Total FLOPs
6.30e+24
Math & numbers · Frontier
AI compute
AI compute
calculator
Plan a training run the way labs sketch one: parameters and tokens in, total FLOPs, GPU-hours, wall-clock time and cost out, first-order estimates, computed live in the page.
6NDtraining-run estimator
B
B
TFLOPS
%
$/GPU-hr
6 × 70B × 15,000B tokens · 1,024 GPUs
$11.06M
GPU-hours
4,423,660
Wall clock
180 days
FLOPs ≈ 6 · N · D · · · GPU-hours = FLOPs / (peak × MFU × 3600)
Doing it by hand too?
A few solid picks. As an Amazon Associate, Calculate.to earns from qualifying purchases, at no extra cost to you.
Casio fx-83GTCW+
The UK classroom-standard scientific calculator (also in pink), allowed in most UK exams.
View on Amazon →
Helix Oxford Scientific
240-function scientific calculator for KS3–KS4 maths, with hard cover. Exam-friendly, plastic-free packaging.
View on Amazon →
OSALO Desktop Calculator
Extra-large 12-digit LCD, giant buttons, solar + battery, the classic office desk workhorse.
View on Amazon →
ROATEE Construction Calc
Feet-inch-fraction engineering calculator for carpenters, framers and contractors on site.
View on Amazon →
The arithmetic every training run starts with
Before any cluster is booked, a language-model training run is sketched with one line of arithmetic: total compute ≈ 6·N·D, where N is parameter count and D is tokens seen. The 6 covers the forward pass (≈2ND) plus the backward pass (≈4ND). This tool takes that number and converts it into the quantities that actually get budgeted, GPU-hours, wall-clock days at your cluster size, and dollars at your hourly rate.
The chain
FLOPs = 6·N·D
GPU-hours = FLOPs ÷ (peak × MFU × 3,600)
days = GPU-hours ÷ GPUs ÷ 24 · · · cost = GPU-hours × rate
GPU-hours = FLOPs ÷ (peak × MFU × 3,600)
days = GPU-hours ÷ GPUs ÷ 24 · · · cost = GPU-hours × rate
MFU, model FLOPs utilisation, is the honesty factor. No real run sustains a GPU's datasheet peak; well-engineered large-scale training typically lands in the 30–50% range once communication, memory stalls and stragglers are paid for. The default of 40% is a sensible planning figure; drop it to 30% for a pessimistic bound or push toward 50% for a highly tuned stack.
Worked default
70B params × 15T tokens → 6.3×10²⁴ FLOPs
at 989 TFLOPS · 40% MFU · 1,024 GPUs · $2.50/hr
at 989 TFLOPS · 40% MFU · 1,024 GPUs · $2.50/hr
Honest limits: 6ND is a first-order rule. It omits attention's context-length term, activation recomputation, embedding maths and any restarts, real bills usually land somewhat above it, and Chinchilla-style scaling questions of what N and D should be are a separate decision this tool deliberately stays out of. Treat the output as the right order of magnitude and a fair comparison between plans, not a quote. The inference line of thinking, roughly 2N FLOPs per generated token, lives in the FAQ below.
Common questions
AI compute FAQ
Each token in training costs about 2N FLOPs on the forward pass and about 4N on the backward pass, so ≈6N per token, times D tokens. It is the standard first-order estimate used across scaling-law work; real runs add overheads on top, but 6ND sets the order of magnitude every plan starts from.
Model FLOPs utilisation: the fraction of the GPUs' theoretical peak your training loop actually sustains. Communication, memory bandwidth and stragglers all eat into it. Large well-tuned runs typically report 30–50%; 40% is a reasonable planning default, and using 100% is the classic way to underestimate a budget by 2–3×.
The dense (non-sparse) throughput for the precision you train in, for example an H100 SXM is roughly 989 TFLOPS at BF16 dense. Marketing sheets often quote the 2× sparsity figure; training does not get that, so use the dense number or your estimate will be optimistic by half.
A decent rule of thumb is ≈2N FLOPs per generated token. Divide your GPU's effective throughput (peak × an inference MFU, often 20–40%) by 2N for tokens per second per GPU, then scale by fleet size and utilisation. Batch size, KV-cache memory and context length move the real number substantially.
Because 6ND is the floor, not the invoice: attention adds a context-length term, activation checkpointing recomputes work, evaluation and checkpoint saves burn time, and few runs finish without a restart. Teams commonly pad the estimate by 20–50%. The tool's job is the defensible baseline those pads are added to.
Keep going