What AI should not be allowed to do in carbon accounting

Every sustainability platform is advertising AI right now. Almost none of them describe what their AI is prevented from doing.

That seems like the more useful thing to publish, so here is ours — including the case where we got the boundary wrong and had to move it.

The one rule everything else follows from

The model never does arithmetic.

Not "the model's arithmetic is checked". Not "the model is good at arithmetic now". It does not do it at all. Every number in the system comes from a deterministic engine operating on approved data, in exact decimal arithmetic, producing the same answer every time it runs.

What the model does instead is judgement work: which GHG Protocol category this activity belongs to, which abatement measure suits this site, how to write a boundary description around figures it was handed. Those are genuinely hard, genuinely useful, and none of them require it to multiply anything.

The reason is not that models are bad at arithmetic. It is that a carbon inventory is a document somebody signs, and a figure in it needs to be reproducible by someone who does not have the model. If a number came out of a language model, its provenance is a prompt and a random seed. If it came out of the engine, its provenance is an activity record, an emission factor, a unit conversion and a global warming potential — all of which are on screen, and all of which an assurance provider can re-derive.

Where we got it wrong

We wrote a feature that drafts the narrative sections of a report — the boundary description, the methodology statement, the year-on-year commentary. Those sections cannot be derived from data, which is why most inventories have correct numbers and three empty pages.

The first real run produced this, in the performance section:

Total emissions decreased by 3,747 tCO₂e, a reduction of 49.2% against FY2025 (FY2025 total = 7,616.988 tCO₂e).

The actual FY2025 total was 4,127. Every figure in that sentence was invented — five of them, all fluent, all in the right units, all wrong.

Our fact checker caught them. It scans every number in generated prose and matches it against the figures the model was actually given; anything unmatched is flagged on the section with the surrounding sentence quoted. So the failure was visible before anyone could accept it.

But the checker catching it was not the interesting part. The interesting part was why the model had done it.

We had asked the section to compare the current year against the previous one, and we had not put the previous year's figures in the brief. The model was asked to perform a comparison it had no way to perform. Faced with that, it did what a language model does: it produced text of the right shape.

The fix was not a better checker, and it was not a sterner instruction. It was noticing that we had asked a language model to do subtraction. Now the system computes the year-on-year change — the absolute movement, the percentage, the direction — and hands those to the writer as stated facts. The model restates them. It no longer calculates anything, and that section has been clean since.

The rule was right. We had quietly broken it in one place and not noticed until it produced something obviously false.

Closed vocabularies

The second constraint is that the model chooses from lists rather than generating freely.

Asked openly which category an emission belongs to, a model will produce "Employee Travel" or "Category 6a" — plausible-sounding labels that appear in no framework. Asked openly for reduction measures, it produces "optimise operations" and "engage stakeholders", which nobody can cost, schedule or verify.

So it picks from the actual GHG Protocol taxonomy: four Scope 1 sub-categories, four Scope 2, and the fifteen numbered Scope 3 categories. It cannot invent a sixteenth. Abatement measures come from a fixed library where each one carries a credible reduction range and the reasoning behind that range — because the percentages vendors quote for lighting retrofits are per fitting, not per site, and a model reading marketing material will cheerfully suggest a lighting upgrade will halve a factory's footprint.

The vocabulary also lets us catch a specific kind of confident error. A model can return "Scope 1" alongside "business travel" — each field valid on its own, the combination flatly wrong. That reads as authoritative and it is exactly the sort of thing a reviewer skims past. The system rejects it outright and records nothing.

Nothing commits itself

Every AI output in the system is a proposal.

A classification lands in a review queue with its reasoning and the lookups it made. A drafted report section is a draft that cannot reach the exported PDF until a person accepts it. A supplier's submitted figure becomes a draft activity record, reviewed like anything typed in-house. A questionnaire answer pre-filled from the inventory is marked as pre-filled and needs someone's name against it before it counts.

This is not ceremony. It is what makes it acceptable for the model to be wrong sometimes — which it will be, because the boundary questions in Scope 3 are ones that experienced practitioners disagree about. An assistant that is right 90% of the time and always reviewed is useful. The same assistant writing directly into the inventory is a liability, and the difference is entirely in the workflow rather than in the model.

We also require a reason when someone rejects a suggestion. A dismissed proposal with no explanation gets proposed again next quarter and dismissed again, and nobody learns anything.

Where we deliberately did not use AI

A customer asked us to pull figures from the carbon inventory into their ESG questionnaires automatically, and suggested AI for the mapping.

We looked at the question bank. Of forty numeric questions, nine can be answered from carbon data at all — the rest ask about water, headcount, governance, training hours and money. Nine known question codes is a lookup table, not a machine learning problem, and a table is exact, instant and reviewable. A model would have added latency, cost and a failure mode, in exchange for nothing.

It is worth saying plainly because the incentive runs the other way: there is commercial pressure to put AI into every feature and describe it that way. The mapping is a hard-coded table of thirteen entries. We kept the model for the place it genuinely earns its keep — the narrative questions, where figures have to be composed into a sentence, and where it is instructed to decline more often than it answers.

The constraint we are most pleased with

On those narrative questions, the model is told:

Asked "are you applying an internal carbon price?", it declines, because that is not in the inventory. Asked "provide your sustainability metrics and targets", it answers with the actual scope figures — and opens with "No emissions reduction target has been recorded in this system", because there isn't one, rather than dressing the absence up as ambition.

A declined question is recorded as considered-and-unavailable, sitting next to the questionnaire, so it reads as a known gap rather than something forgotten.

Why publish the constraints rather than the capabilities

Because in this domain the capabilities are converging fast and the constraints are not.

Any platform can extract a figure from a PDF this year. What differs is whether the number that comes out can be traced, whether it waits for a human, whether the system says so when it is unsure, and whether anyone thought about what happens when the model is confidently wrong.

Those are design decisions, not model capabilities. They do not improve when the next model ships. And they are the ones that determine whether the resulting inventory survives being asked about.