How Much Calculation Produces a Token?
A small multiply-and-add example explains why active model weights give a useful first estimate of computation, but not a complete estimate of electricity.
The phrase “seventy-billion-parameter model” sounds like it ought to tell you how expensive the model is to run. It gives you a useful clue. Before it becomes an energy estimate, however, you need to know which parameters take part in the calculation and what the hardware does with them.
A parameter is a learned numerical value. Many model parameters are arranged in matrices: rectangular tables of numbers. During generation, the computer combines those tables with numerical representations of the text. The large operation counts begin with repeated multiplication and addition, which can be understood without first understanding the entire model.
Scope note: the estimates below count conventional arithmetic for learned linear transformations. Attention, memory transfers, and other operations must be added before considering a complete execution.
One small calculation, repeated
Take three input numbers, 2, 3, and 4. Multiply them by three weights, 5, 6, and 7, then add the results:
(2 × 5) + (3 × 6) + (4 × 7) = 56.
That expression uses three multiplications and two additions. If another row of weights needs a result, repeat the work with that row. A matrix with ten rows and three columns requires thirty multiplications and twenty additions in this simple evaluation.
For long rows, the difference between the multiplication count and addition count becomes relatively small. Engineers therefore use an approximate count of two operations per weight: one multiplication and one addition. Some hardware performs both together in a fused multiply-add instruction, but the familiar performance convention still counts the pair as two arithmetic operations.
Adding up the weights in the participating matrices gives a useful leading estimate. A dense model using roughly seventy billion matrix weights performs about 140 billion arithmetic operations per generated token for those transformations. “Dense” here means that the relevant weight sets generally participate, rather than a router selecting only certain expert sections for each token.
This is a calculation from the model’s structure, not a claim that someone measured 140 billion separate instructions on a processor. It also does not require every conceivable algorithm to evaluate the model in exactly this way. It describes the conventional work being estimated.
The transformer architecture includes attention as well as learned linear transformations, so the 140 billion figure is a starting term. The next article adds a context-dependent attention estimate. Treating the first term as the whole calculation would lose the very effect that makes long conversations interesting.
A large model may use only part of itself
Some models divide portions of their network into specialized sections called experts. A routing mechanism selects a subset for each token. This arrangement is called a mixture of experts; it lets the model contain more parameters than it activates in any one token’s calculation.
DeepSeek’s V3 technical report describes 671 billion total parameters, with 37 billion activated per token. Applying the same rough two-operations-per-active-parameter estimate gives 74 billion operations, rather than the 1.342 trillion obtained by treating every parameter as active.
That is a large difference produced by a small wording distinction. “Total” tells you how many parameters the model contains. “Active” tells you which portion participates in the token’s calculation. A comparison that drops the adjective can make the model appear to perform far more arithmetic than it ordinarily does.
The inactive weights have not ceased to exist. The deployment still needs some arrangement for storing and accessing experts, and spreading those experts across processors can require communication that the active parameter count does not describe, even before the routing work is considered. The active count helps explain the arithmetic; the actual execution determines which data moves and where.
For a reader comparing models, this is a useful habit. Keep the total count, active count, numerical format, and architecture attached to the name. A larger advertised parameter count alone cannot establish a larger electrical cost per output token.
Why the processor brochure cannot finish the sum
Once there is an operation count, it is tempting to divide a processor’s power rating by its advertised operations per second and declare a result. The units do work. The operating assumptions need inspection.
NVIDIA specifies the H100 SXM at up to 700 watts. Its dense FP8 arithmetic reference is about 1.979 quadrillion operations per second, taking the sparsity convention in the published specification into account. FP8 is an eight-bit floating-point number format; sparse performance figures assume a supported pattern of omitted work and should not be substituted for the dense figure.
Dividing those nameplate quantities gives approximately 0.354 picojoules per operation. Multiplying by 140 billion operations produces about 0.0495 joules. That is a conditional peak-throughput reference, calculated from specifications. It is not a measured energy cost for a seventy-billion-parameter model.
Actual execution may fail to achieve peak arithmetic throughput, and power need not equal the maximum setting. If the arithmetic circuits wait for values to arrive while memory transfers and attention operations are still in progress, a calculation based on continuously sustained peak arithmetic will misdescribe the run.
There is another precision detail worth keeping visible. Seventy billion weights stored at one byte each occupy about seventy gigabytes before extra metadata and other runtime storage. At two bytes each, they occupy about 140 gigabytes. A weight count does not determine either storage size without a representation assumption.
The same H100 specification lists memory bandwidth of 3.35 terabytes per second, so streaming seventy gigabytes of weights once would take at least about 21 milliseconds even if the system sustained that peak rate throughout the transfer. The arithmetic units cannot use values that have not arrived. Some transfers and calculations can overlap, but a useful time estimate still has to respect the slower requirement, together with any dependencies that prevent the work from overlapping.
Keep the arithmetic and the electricity separate
I would retain the operation estimate because it explains what the system is being asked to calculate. It helps compare architectures, catches the mistake of treating every expert as active, and makes it possible to ask whether computation or data movement is likely to constrain execution.
The estimate becomes misleading only when it is promoted into a complete energy measurement without the missing information. Operations must be executed somewhere, at a particular precision and speed, while the necessary values move through memory and between devices.
The same model can also perform different amounts of attention work as its input grows. A brief question and a conversation containing a long report may end with equally short replies, yet require different calculations for each new token. The stored conversation is the next quantity to examine.
