Where Does the Electricity Go?
Generating text consumes energy in switching circuits, moving numbers, and keeping hardware powered. Arithmetic alone leaves out much of the work.
A processor can spend energy fetching a number before it does anything useful with that number. It can then spend more energy on the calculation, on moving the result, and on remaining powered while other parts of the system catch up. Counting multiplications tells you something important about an AI model. It leaves several electrical activities uncounted.
In the first article, I separated a token’s electricity use from its selling price. Now the question is what actually consumes that electricity. The answer starts with ordinary electronic behavior, though it becomes difficult to follow when billions of operations and many layers of memory are involved.
Scope note: this article explains the physical mechanisms. The historical examples illustrate relative costs; they are not specifications for a current GPU.
Changing an electrical state
Digital circuits represent information through electrical states. Switching between states involves charging and discharging small capacitances: parts of the circuit that hold electrical charge. Repeat that activity across a processor and the energy use becomes measurable.
A common approximation for this switching power multiplies the amount of switching activity, the effective capacitance, the square of the supply voltage, and the clock frequency. The notation is compact—P ≈ αCV²f—but the useful relationship here is the squared voltage. If voltage falls by ten percent while the other terms remain the same, this switching term falls to 81 percent of its previous value.
That is a 19 percent reduction in one component of energy use. It does not mean every processor can simply lower its voltage and preserve its speed or reliability. It shows why unchanged physical laws permit changed electrical consumption. Different operating conditions produce different amounts of work and dissipation under the same laws.
There are other contributions. Leakage current flows even when a transistor is not making a useful transition. Clock distribution and control circuitry also require power. Mark Horowitz discusses these circuit and data-movement costs in his 2014 engineering paper, Computing’s Energy Problem.
The resulting heat is familiar to anyone who has heard a laptop fan accelerate during a demanding task. A fan’s noise does not measure the energy used by the model, but the cooling system has a real job. Electrical energy dissipated in the equipment must be removed if the hardware is to keep operating within its temperature limits.
Fetching the number can cost more
Horowitz’s paper includes a useful historical comparison. Under its 45-nanometre, 0.9-volt examples, a 32-bit floating-point multiplication used roughly 3.7 picojoules. A 64-bit access to DRAM, a type of main memory, used around 1.3 to 2.6 nanojoules.
A nanojoule is a thousand picojoules. The memory access in that example therefore consumed hundreds of times the energy of the multiplication. The operations have different widths and purposes, so this is an illustration of scale rather than a like-for-like specification. More importantly, these are old engineering figures. Using them as coefficients for a current GPU would produce a misleading estimate.
The lasting point concerns the arrangement of the work. A number already held in a small storage area beside the arithmetic circuits can be cheaper to use than one that must be fetched from farther away, which gives the implementation a practical reason to preserve and reuse nearby values. Repeatedly transferring those values can add energy consumption without changing the calculation’s answer.
This is a practical software issue. The FlashAttention paper, for example, organizes attention calculations to reduce transfers between high-bandwidth memory and on-chip storage. The method preserves the mathematical attention operation while changing how it is executed. A later article will examine why such changes can matter without reducing the model’s parameter count.
For now, imagine inspecting an actual execution record. You would want the arithmetic count, but you would also want to know which values were read, from where, how often they were reused, and which results had to be written back. All of those actions occur in physical hardware.
Time contributes to energy
A slow operation does not become free because little arithmetic is happening. The hardware remains powered during the wait. Memory, links between processors, control circuits, and other components may remain active even when the main arithmetic units are poorly used.
This creates an important distinction between power and energy. A device drawing 500 watts for one second uses 500 joules. A device drawing 300 watts for two seconds uses 600 joules. The second device has lower power during the example but consumes more energy to finish the work.
The numbers are hypothetical, though the relationship between them follows directly from the units. To compare energy, you need the rate of consumption and the duration together, so a lower instantaneous power reading, a faster response, or a smaller processor cannot independently settle which configuration uses less energy per completed task.
The measurement can be taken at different places. GPU sensors cover a different boundary from a meter supplying the whole server. A facility measurement adds further electrical loads, including cooling and power conversion. Google’s study of AI serving explicitly includes such full-stack considerations. A GPU result cannot silently stand in for that wider measurement.
The physical minimum is a different calculation
There is a deeper thermodynamic question. Landauer’s principle gives a minimum dissipation for the standard case of irreversibly erasing an unknown, equally likely binary value. At 300 kelvin, the expression kT ln 2 is about 2.87 × 10⁻²¹ joules per erased bit. The Boltzmann constant supplies k; Landauer’s original paper establishes the information-erasure argument.
That limit is extraordinarily small, but it cannot be multiplied by the number of characters in an answer to calculate the energy used to produce it. A character is not a bit erasure. Nor is a floating-point multiplication one bit erasure. The model performs internal calculations that are not described by the size of its final output.
The distinction matters because a fundamental lower limit and a real processor measurement answer different questions. The first concerns specified irreversible transformations under thermodynamic assumptions. The second concerns an actual machine executing an actual workload, with memory movement, leakage, timing, and control included.
For a present-day cost estimate, the second is the useful starting point. Before measuring all of it, though, it helps to count the arithmetic the model asks the processor to perform.
