Scaled biogas pilot: 5 quantitative success criteria

In the biogas sector, a pilot without quantitative criteria defined beforehand is not a pilot: it is a commercial demonstration dressed up as technical validation.

The difference between the two is making decisions with data or making them with perceptions.

This article describes the five KPIs that define the success of a pilot in a biogas plant, the transition rules from batch to continuous, the safety factors when scaling from pilot to industrial, and the most frequent patterns that ruin the investment decision.

Why a biogas pilot without quantitative criteria is not a pilot

A technical pilot answers a binary question: does the proposed intervention improve the process in a measurable, reproducible and economically justifiable way, yes or no?

To answer it, three things are needed before starting:

• An adequate control: the plant or reactor under nominal conditions, without the intervention.
• A quantitative hypothesis: what magnitude of improvement is expected and through which mechanism.
• Decision thresholds: what minimum value of the KPIs determines that the result counts as a success.

Without those three elements, any observed difference can be attributed to natural process variability, analytical noise or operator bias.

In practice, most commercial pilots are run without them. The supplier installs their technology, waits for a visible improvement and builds the success narrative after the fact on whatever data supports their case.

This is known as commercial p-hacking: the experiment is run and only then is it decided which metric defines success. The consequence is predictable: the client pays for an intervention that does not replicate when scaled to stable industrial operation, because the observed improvements were not real but favourably selected variability.

On top of that, a certain stability over time in the operating and feeding parameters has to be ensured.

The 5 quantitative KPIs of the Smallops scaled biogas pilot

The following five criteria are the minimum system:

• They are defined and signed off before starting the pilot.
• They are measured at a minimum weekly frequency throughout the period.
• They are assessed collectively at the end: at least four of the five must be met to authorise industrial scale-up.

#	KPI	Minimum success threshold	Measurement method
1	Specific productivity	≥ 7% increase over the control, at equal OLR and diet	Weekly Y CH₄ in Nm³ CH₄/kg VS fed; comparison with the baseline of the previous 30 days
2	Operational stability	FOS/TAC < 0.35 during 80% of the period	Weekly Nordmann method; log of excursions
3	Gas quality	≥ 30% reduction in H₂S (when the scenario includes sulphide mitigation)	Colorimetric tubes or electrochemical sensor; weekly reading
4	Reproducibility	Effect sustained for at least 2 HRT without new dosing	Extended monitoring after the initial dosing
5	Economic	Projected 12-month ROI ≥ 1.8x on the total cost	TCO model including additives, analyses, labour and possible shutdowns

KPI 1 · Specific productivity: why ≥ 7% and not more

The 7% threshold may look modest, but it responds to three statistical realities:

• The natural variability of specific productivity between weeks, in stable plants, is around 4-6% (coefficient of variation of weekly Y CH₄).
• The accumulated analytical uncertainty of the method (VS characterisation + flow measurement + compositional analysis) adds another 2-3%.
• The operator motivational bias (better care of the digester during the pilot) usually contributes a further 2-3%.

A 5% improvement can be noise. A well-measured 7% already exceeds the detection threshold and allows the effect to be attributed to the intervention.

KPI 2 · Operational stability: FOS/TAC as the anchor

A one-off productivity improvement without stability is not a useful improvement: if you recover 8% of the CH₄ yield but the digester enters the alert zone (FOS/TAC > 0.4) several times during the pilot, what you gain in methane you lose in operational risk and reactive corrections.

The threshold of 80% of the time with FOS/TAC < 0.35 allows acceptable one-off peaks without penalising the overall reading. The full reading of this variable is detailed in stabilising the anaerobic digester.

KPI 3 · Gas quality: only if applicable

H₂S reduction is a conditional KPI: it is only measured when the operational scenario includes sulphide mitigation as part of the intervention (typically when the diet has a high sulphate content or sulphur in the molecular structures, or when the plant cogenerates with an engine sensitive to H₂S).

In pilots focused on methanogenesis kinetics, with no gas quality component, this KPI does not apply and the criteria become four: success then requires meeting three out of four.

KPI 4 · Reproducibility: 2 HRT without dosing

Some interventions produce an impressive initial peak that disappears after 2-3 weeks.

Extended measurement over 2 HRT without new dosing separates two cases: the genuine kinetic effect (which persists as long as the consortium stays adapted) and the transient effect (which requires continuous dosing and therefore permanently inflates the OPEX).

This criterion rules out the solutions that only work while the supplier keeps invoicing you for additive.

KPI 5 · Projected ROI: the economic threshold

A 12-month ROI of 1.8x is the minimum that justifies the scale-up decision, given the operational risk of changing the stable regime of the plant. A lower ROI does not mean the intervention is technically bad, but it does not compensate for the risk of modification.

The TCO model must include:

• The cost of the additive or the intervention.
• Reinforced analyses during adaptation (typically 60-90 extra days of monitoring).
• Additional labour.
• A buffer for possible minor shutdowns during the transition.

Scaling from batch to continuous: transition rules

The first scaling step happens between the batch BMP test (the theoretical upper bound of the substrate) and the semi-pilot reactor in continuous mode.

That semi-pilot reactor is, in practice, a semi-continuous test: the laboratory reactor is fed continuously over several weeks, with the load and retention time of the target digester. Against the BMP, which measures the maximum potential under ideal conditions with a single load, the semi-continuous test reveals the kinetics, the tolerance to load, the inhibitions and the acclimation time. It is the most reliable validation before scaling up, and the one that prevents arriving at the industrial pilot with unverified assumptions.

Three operational rules govern this transition:

• The 70-80% rule: the specific productivity to be expected in semi-continuous mode is typically 70-80% of the BMP of the substrate (measured under VDI 4630). If your BMP gives 350 NmL CH₄/g VS, expect 245-280 NmL CH₄/g VS in continuous operation. Expecting more is ignoring the thermodynamics of the process.
• The equivalent HRT rule: the semi-pilot reactor must operate with a hydraulic retention time (HRT) equivalent to that of the target industrial plant, not with the HRT that maximises production in the pilot. Testing with an HRT of 60 days and then scaling to 30 in the industrial plant usually gives dreadful results.
• The real diet rule: the pilot diet must replicate the real diet of the plant, including its seasonal variability. Testing with a selected homogeneous substrate and then facing variable substrate in the industrial plant is a classic anti-pattern.

Scaling from pilot to industrial plant: safety factors

The second step, from semi-pilot to industrial digester, multiplies the volume by two to four orders of magnitude. Biochemical kinetics does not scale linearly with volume.

Factor	Magnitude	Why it matters
OLR · organic loading rate	Apply 80-85% of the OLR validated in the semi-pilot	Mass transfer worsens in large geometries, especially if the mixing is not maintained
HRT · hydraulic retention time	Maintain at least 100% of the semi-pilot HRT	Reducing the HRT when scaling worsens the digestibility of lignocellulosic fractions
Additive dosing	Start at 60-70% of the dose validated in the pilot	Additive distribution is worse in large geometries; better to underdose and increase gradually

The three factors are applied simultaneously during the first 4-6 weeks of industrial operation.

Once stability is confirmed when scaling up (FOS/TAC < 0.35, specific productivity within the expected range and no propionic acid accumulation), they can be relaxed gradually, one by one, until reaching the parameters validated in the pilot.

Typical negative decisions in commercial pilots

• Pilot without a control: comparing production during the pilot with that of the same digester a month earlier, ignoring that the diet or the seasonality may have changed. Without a simultaneous control (another reactor, or the digester itself with a clean baseline), the result is not attributable.
• Cherry-picking of KPIs: measuring 15 variables, showing the 3 that improve and omitting the 12 that worsen or stay the same. An effective intervention improves a coherent subset of related variables (productivity + stability), not random ones.
• Pilot too short: pilots of 30 days or less. Biogas kinetics has a memory of weeks; transient effects can dominate and mask the real steady-state behaviour.
• Over-caring for the pilot: during the pilot the operator pays extra attention to the digester (daily checks, fine load adjustments, a more stable diet). The observed improvement is partly due to the care, not the intervention, and it disappears on scale-up.
• No real TCO: presenting productivity improvements without including the total cost. A 10% improvement in Y CH₄ that costs 12% of the value of the biogas produced is a net loss.

Operational case: pilot in an 800 kWe agro-industrial plant

An 800 kWe plant (mesophilic, 38 °C) with a mixed diet: 60% pig slurry, 30% maize silage and 10% fruit waste.

Hypothesis: application of iron nanoparticles in a carbon matrix to recover the specific productivity lost after an unstable season.

Pilot design: 12 weeks of baseline (control), 6 weeks of dosing at 2.5 g Fe/kg VS fed, and 12 weeks of follow-up without re-dosing.

Result assessed against the 5 KPIs

Frequently asked questions about scaled biogas pilots

How long should a biogas pilot last to be valid?

A valid pilot requires a minimum of 3 complete HRT after the initial dosing: one for stabilisation, one for active measurement and one for confirmation without re-dosing.

For digesters with an HRT of 20-25 days, this amounts to 60-75 days of active pilot, plus the prior baseline period (minimum 30 days). Pilots of less than 30 days in continuous mode do not allow the genuine kinetic effect to be separated from transient effects.

Why do commercial biogas pilots fail?

Through four recurring patterns: lack of a control (there is no simultaneous reactor or baseline to compare against, so the improvement is not attributable) and cherry-picking of KPIs (many variables are measured and only the ones that improve are shown).

And two more: a pilot that is too short (below 30 days transient effects dominate) and the absence of a real TCO (productivity improvements are presented without counting the total cost, and a 10% improvement that costs 12% of the value of the biogas is a loss).

What safety factor should be applied when scaling the OLR?

When scaling from pilot to industrial plant, apply the OLR at 80-85% of the one validated in the pilot during the first 4-6 weeks.

Mass transfer worsens in large geometries (effective mixing, greater spatial heterogeneity), so the same load that was stable in the pilot can generate VFA accumulation in the plant. After 4-6 weeks of stable operation, raise it gradually to 100%.

How is a pilot result transferred to an industrial plant?

By applying three safety factors during the first 4-6 weeks: starting at 80-85% of the validated OLR, maintaining at least 100% of the pilot HRT and dosing at 60-70% of the validated dose.

Once stability is confirmed (FOS/TAC < 0.35 and productivity within the expected range), the factors are relaxed one by one until reaching the pilot parameters.

Is a semi-continuous test needed before setting up the pilot?

If the decision depends on the kinetics or on the tolerance to load, yes. An industrial pilot is expensive and slow; a semi-continuous test in the laboratory costs a fraction and answers in advance the questions that carry the most risk: how fast the substrate degrades, how much load it tolerates, whether inhibitions appear and how many weeks the consortium needs to acclimate.

The BMP does not answer that, because it is a batch test and measures the maximum potential under ideal conditions. The sensible sequence is BMP to screen, semi-continuous to validate and pilot to confirm at scale.

Normative and bibliographic references

VDI 4630 (2016). Fermentation of organic materials. Verein Deutscher Ingenieure.

Holliger, C. et al. (2016). Towards a standardization of BMP tests. Water Science and Technology, 74 (11), 2515-2522. → doi.org/10.2166/wst.2016.336

Pohl, M. et al. (2012). Anaerobic digestion of straw: effect of biological pre-treatment. Bioresource Technology, 124, 354-360.

Angelidaki, I. et al. (2018). Biogas upgrading and utilization: current status. Biotechnology Advances, 36 (2), 452-466. → doi.org/10.1016/j.biotechadv.2018.01.011