The Number Is Not Enough

A measurement usually arrives as a row in a file

A facility was surveyed. The vendor found a plume and reported 12 kg CH₄/h. Or perhaps the file contains no plume at all.

The row looks like data. It has a timestamp, a location, and a number with units. It may even include a confidence interval. Surely that is enough to place it into an emissions inventory.

It is not.

To use the observation in inference, we must be able to ask how probable that exact vendor outcome would have been under every candidate emissions state. How likely was the system to report a 12 kg/h plume if the source was truly 5 kg/h? What if it was 20 kg/h? If nothing was reported, which source rates could realistically have escaped detection under that pass’s conditions?

Those questions require a likelihood. Without one, the number cannot tell us how strongly to favor one emissions hypothesis over another.

A sample is defined by its likelihood

Let $Q$ be the true emissions rate, $x$ the measurement conditions, $s$ the sensor system and algorithm version, $D$ the detection indicator, and $Y$ a reported rate. Define $p_s(Q,x)$ as the probability that system $s$ reports a detection under rate $Q$ and conditions $x$.

For a pass that returns no report, the likelihood contribution is

For a pass that reports rate $y$, the joint likelihood is

These are the two branches of the same observation model. The first needs a probability-of-detection function. The second needs that function and a conditional quantification-error distribution.

A threshold such as “minimum detection limit: 5 kg/h” supplies neither branch. It does not say whether a 5 kg/h source is detected 5%, 50%, or 95% of the time. A statement such as “accuracy: ±30%” does not identify a probability distribution, confidence level, bias, tail behavior, operating conditions, or the selection process that determined which cases were reported.

Those claims may summarize performance. They do not define evidence.

See what happens when the kernel is missing

The interactive simulator compares two historical, paper-derived aerial likelihood examples with an intentionally incomplete threshold-only datasheet.

Choose Rate reported or No report, then hold the hypothetical true rate and conditions fixed. The first two technology contracts can score the observation. The threshold-only contract cannot—not because its technology is necessarily poor, but because the description withholds the probability model needed to interpret its output.

This is not a vendor leaderboard. The two published functions come from different systems operating at different paper-era geometries and protocols. Their purpose here is to demonstrate that each technology has its own observation law. The same reported number does not carry the same evidential weight under every sensor.

The comparison also shows why detection and quantification cannot be collapsed into one “accuracy” metric. A technology may be unlikely to detect a particular source, yet produce a reasonably concentrated rate estimate when it does. Another may detect the source more readily but return a wider conditional range. Those are different information profiles.

What a vendor needs to provide

The goal is not to demand that every vendor expose proprietary retrieval code. The goal is to obtain the calibrated input-output behavior needed to evaluate the delivered evidence.

At minimum, a usable measurement package needs the following components.

1. A versioned probability-of-detection function

The vendor should provide

where $s$ identifies the technology and algorithm version. The function should identify every material conditioning variable used in calibration: wind, altitude or standoff, scene class, surface reflectance, instrument configuration, source geometry, or others supported by the evidence.

The calibration data should include both detections and non-detections from controlled releases. The validity envelope must travel with the curve: tested rate range, environmental range, operating geometry, site types, and known exclusions. A curve outside its tested envelope is an extrapolation, not a calibration.

Continuous POD functions and uncertainty models can be estimated from blinded controlled-release data; Conrad, Tyner, and Johnson demonstrate this for three airborne systems. Their work also shows why conditions such as wind and flight altitude belong inside the function rather than in a footnote.

2. Quantification error conditional on detection

Given that a report was produced, the vendor should provide

For multiplicative errors, a lognormal model is often a useful representation:

The deliverable should disclose bias, spread, heteroskedasticity, tails, and correlated error—for example, a wind-product bias shared by every plume quantified on the same flight day.

Most importantly, the quantification law must state that it is conditional on detection and on the reporting protocol. Near the detection boundary, reported cases can be a selected subset of all attempts. Mixing a POD curve from one experiment with a quantification model fitted under a different selection rule can create a joint likelihood that corresponds to no actual measurement process.

3. Coverage and evaluability for every opportunity to observe

The absence of a plume is informative only where the technology had a documented opportunity to see one.

Each pass or exposure should identify:

• which facilities, domains, or source groups were covered;
• which were evaluable;
• the conditions used to evaluate POD for each covered unit; and
• why any nominally covered unit was not evaluable.

An evaluable non-detection contributes $1-p_s(Q,x)$. A non-evaluable pass contributes no evidence. If a vendor delivers detections but omits the negative coverage record, most of the survey’s information cannot enter the inventory.

4. The reporting protocol

The likelihood must reproduce what the vendor actually reports—not an idealized sensor hidden upstream of business rules.

The protocol should specify:

• reporting and suppression thresholds;
• treatment of partial detections and quality-control failures;
• whether several observations are merged into one report;
• the exposure denominator for false-positive rates; and
• the meaning of alarm onset and cessation timestamps.

If detected values below a reporting floor are suppressed, that floor changes both the report and no-report branches. It is part of the likelihood.

5. Detection unit, localization, and aggregation semantics

What physical quantity does the kernel act on?

A system may detect spatially resolved plume features, a facility-total aggregate, a satellite pixel, or a vendor-defined merged feature. Those choices are not interchangeable. The package must define:

• the spatial and temporal support of the measurement;
• what counts as one detectable feature;
• whether nearby sources are separable or merged;
• localization uncertainty or a source-confusion matrix; and
• how diffuse baseline emissions enter, if at all.

Without those semantics, an analyst may apply a plume-level POD to an arbitrary sum of sources—or divide a facility-total report among equipment that the observation never resolved.

6. False-positive behavior

A report should not become mathematical proof that a real source existed. Vendors should provide zero-release or negative-control results and define false reports per unit exposure.

The needed object is not merely a false-positive count. It is a rate with uncertainty, plus the distribution of any payload attached to a false report. Observing zero false positives in a limited trial places an upper bound on the rate; it does not establish a structural zero.

7. Calibration uncertainty, provenance, and versioning

A fitted curve is itself uncertain. The vendor or independent evaluator should provide parameter uncertainty, bootstrap draws, a posterior sample, or another machine-usable representation of calibration uncertainty.

Every kernel should also carry:

• the controlled-release methodology and blinding status;
• calibration-data lineage or sufficient reproducibility artifacts;
• model-selection and exclusion rules;
• software and algorithm version;
• effective dates; and
• a change log describing whether a new pipeline changes POD, quantification, localization, or reporting behavior.

An annual estimate should be reproducible against the exact measurement model in force when the data were produced.

What is useful—and what is not enough

Procurement should ask for likelihood-ready data

A technically useful data contract can be stated plainly:

1. Deliver every reported payload and every evaluable opportunity that produced no payload.
2. Deliver the covariates used by the calibrated kernel for each observation.
3. Supply versioned POD and conditional quantification functions with uncertainty and validity envelopes.
4. Declare reporting thresholds, quality-control rules, support, grouping, and localization semantics.
5. Provide negative-control evidence for false-positive behavior.
6. Preserve links to raw artifacts and the exact algorithm version that produced each record.

Machine-readable functions or parameter draws are far more useful than a PDF chart. A purchaser should be able to evaluate the likelihood at a proposed rate without digitizing a marketing figure or reverse-engineering an undisclosed confidence interval.

Where Arachne draws the boundary

Arachne does not infer a sensor’s performance from the same facility data whose emissions it is trying to estimate. It consumes externally calibrated likelihood kernels from the strongest applicable evidence, whether produced by the vendor or an independent controlled-release program.

Its responsibility is to enforce the kernel’s conditioning and validity envelope, propagate calibration uncertainty, preserve provenance, and report sensitivity when credible sources disagree.

The vendor’s responsibility is not to provide certainty. It is to provide enough calibrated structure that uncertainty can be used honestly.

A number can be copied into a spreadsheet. A likelihood can support an inference.

References

• Bradley M. Conrad, David R. Tyner, and Matthew R. Johnson, “Robust probabilities of detection and quantification uncertainty for aerial methane detection: Examples for three airborne technologies,” Remote Sensing of Environment 288 (2023): 113499.
• Clay S. Bell et al., “Single-blind determination of methane detection limits and quantification accuracy using aircraft-based LiDAR,” Elementa: Science of the Anthropocene 10, no. 1 (2022): 00080.
• Michael J. Thorpe et al., “Deployment-invariant probability of detection characterization for aerial LiDAR methane detection,” Remote Sensing of Environment 315 (2024): 114435.