Models and Validation: From PLS to Transformers, Under GxP

📍 Where we are: Part I · Foundations of Learning in Bioprocess — Chapter 3. The last chapter built the fuel: data that is ready, contextualized, turned into leak-free features, and honestly split by batch. This chapter chooses an engine. We walk the ladder of model families from partial least squares to transformers, asking at each rung how much data it needs and how it must be validated to be trusted with a decision about a medicine — and we argue, against the prevailing hype, that the simplest engine usually wins.

A model is the function that turns the contextualized fuel of the previous chapter into a number a person can act on: a titer from a spectrum, a clone rank from a sequence, a pass/fail from a release panel. Newcomers to bioprocess machine learning arrive expecting the model choice to be the hard, exciting part — which architecture, how many layers, which optimizer. It is not. By the time you have read this chapter you should hold two convictions that run against the grain of the broader field. First, the model family that earns its keep in production bioprocess is overwhelmingly the simplest one that fits — a 40-year-old linear latent-variable method, not a transformer. Second, choosing the model is only half the job: under GxP (the family of Good-x-Practice regulations — Good Manufacturing, Laboratory, and Clinical Practice — that govern regulated drug work) a model is not "good" because its R² is high (R² being the 0-to-1 goodness-of-fit score, where 1 is a perfect fit), it is trustworthy because it was validated, locked, and documented to a standard a regulator will accept. This chapter covers both halves, because in bioprocess they are inseparable.

We keep the same running example the whole series uses. The golden run BATCH-2026-001 carries an SEC monomer purity of 98.611% (the fraction of intact, single-molecule antibody measured by size-exclusion chromatography, where higher is better); its sibling BATCH-2026-004 goes OOS (out of specification — a result outside its written acceptance limit, which triggers a formal investigation) with a host-cell-protein result of 128 ng/mg against a 100 ng/mg spec maximum (host-cell protein is residual protein from the production cell line, an impurity, so 128 exceeding the 100 maximum is a failure — here lower is better). The same in-line Raman spectra (a light-scattering chemical fingerprint read continuously inside the bioreactor in real time, here 701 wavenumber channels used to infer concentrations) and offline assays (lab measurements that come back days later) from the simulator that fed Chapter 2 feed the models here, and the example suite in examples/platform/ml/ provides a runnable PLS-versus-deep head-to-head we will read together, alongside the run_all.py credibility harness that closes the chapter.

The simple version

Suppose you must learn to guess a cake's sweetness from its smell, and you have only six cakes to learn from. You could hire a world-class chef who has tasted a million cakes and can describe a thousand subtle aromas — but with six examples, that vast knowledge has nothing to grip; the chef will "learn" coincidences in your six cakes and be confidently wrong on the seventh. Or you could use a simple rule of thumb — "more vanilla smell, more sweet" — fit to your six cakes. The rule of thumb is less impressive but far more reliable, because it has less room to fool itself. That is the entire argument of this chapter's first half: a model with fewer ways to go wrong needs less data to go right, and bioprocess never has enough data to feed the fancy model. The second half adds the part no cake has: before anyone is allowed to act on the model's guess about a medicine, the guess — and the model that made it — must be proven, frozen, and written down, so that the guess you trust tomorrow is the one you proved today.

What this chapter covers

• The model families, and when each fits: linear regression, PLS, and PCA (the small-data workhorse); tree ensembles and gradient boosting; Gaussian processes with Bayesian optimization; and neural networks (MLP, 1D-CNN for spectra, autoencoders/VAE, transformers) — each with the core math sketched and the bioprocess question it answers.
• Why deep learning rarely beats PLS on bioprocess data: the bias-variance trade-off, sample efficiency, extrapolation, and interpretability for review — with a runnable PLS-versus-CNN head-to-head, and the real numbers, that shows it.
• The GMP validation paradigm: locked versus continuously-learning models, the Predetermined Change Control Plan (PCCP), GAMP 5 and Computer Software Assurance (CSA) applied to ML, the ISPE GAMP AI guide, the FDA 7-step model-credibility framework, and the draft EU/PIC/S GMP Annex 22 — at the altitude this chapter needs, with the regulation chapter and MLOps chapter going deeper.
• The evidence-tier and maturity callout this book uses to grade every claim, formally introduced here.

How this book grades its evidence

Before we compare models, we fix the lens this book uses to judge every claim about them — because the literature on bioprocess ML is full of impressive numbers whose credibility varies wildly. The same headline ("our model cut experiments by a third," "first in-line release," "95% auto-released" — release being the formal quality decision to let a finished batch be used or sold) can be a peer-reviewed result, a vendor slide, or a press release, and the gap between those is the difference between a fact and an advertisement. From here on, every external claim carries two labels, kept deliberately separate.

The first is an evidence tier, ranked from strongest to weakest:

• peer-reviewed-independent — published, peer-reviewed work whose authors are not the vendor or operator selling or running the thing. The gold standard.
• peer-reviewed-self-authored — peer-reviewed, but co-authored by the company that built or deployed the method. Credible on the science, but the framing favors the author, and the comparison is usually against the author's own prior baseline.
• vendor-self-reported — a vendor's own claim (a white paper, a slide, a product page). Useful as a signal of direction; not evidence of a result.
• press-release-only — a number with no method behind it you can inspect. Treat as marketing.

The second is a maturity label for how far a method has actually traveled: (production) means deployed in a routine GMP plant; (pilot) means demonstrated at scale but not in routine production; (research) means a paper or a lab result. A claim can be high on one axis and low on the other — a peer-reviewed-independent result that is only (research), or a (production) deployment whose only public evidence is vendor-self-reported. Keeping the two apart is the whole point, and it is why this book refuses to launder a vendor slide into an established fact: the suite's case_ledger.py makes the discipline literal, grading sixteen named industry deployments and finding that zero of seven headline numeric claims clear the peer-reviewed-independent floor, so every one of them is hedged on sight.

Evidence

This is the callout you will see throughout the book. Whenever a claim's credibility or maturity is contestable — a headline accuracy number, a "fewer experiments" figure, a "first-ever" deployment — it gets graded here, in its own block, so the tier travels with the claim and is never silently upgraded. The strongest single anchor for this chapter's central argument — that classical chemometrics beats deep learning on real bioprocess spectra — is peer-reviewed-self-authored and (pilot): the Boehringer Ingelheim study that predicted 16 quality attributes (the measurable properties — purity, aggregate, impurity levels — that define whether product is acceptable) in-line during Protein A capture (the first antibody-purification step), whose best overall model was k-nearest-neighbors (a classical method that predicts from the most similar past examples), not a deep network [1]. We return to it below.

The model families, and the question each one answers

There is no single best model. There is a ladder, ordered roughly by how much data and how much trust each rung demands, and the engineering skill is to climb no higher than your problem forces you to. Here is the ladder, rung by rung, with the core idea, the bioprocess question each rung was built to answer, and the cost it carries into a GMP review.

Linear regression, PLS, and PCA — the workhorse

At the bottom sits the family that does the overwhelming majority of real production work in bioprocess: linear latent-variable methods. Principal Component Analysis (PCA) compresses many correlated variables into a few uncorrelated directions of greatest variance; mechanically, it diagonalizes the covariance matrix of X and keeps the top eigenvectors, so a 50-column release panel collapses to two or three scores that carry most of the signal. It is the engine of the multivariate statistical process control (MSPC) the release chapter builds — the suite's mspc.py fits PCA on five passing batches in two components and the model's Hotelling's T² and squared-prediction-error (SPE) statistics flag the one OOS batch cleanly (T² measures how far a batch sits from normal inside the model's space, SPE measures how far it sits outside that space — together they catch both kinds of abnormality), with the SPE contribution pointing straight at the host-cell-protein channel. Partial Least Squares (PLS) is its supervised cousin: where PCA finds the directions of greatest variance in X, PLS finds the directions in X that have the highest covariance with y — the directions that best predict the target, not merely the ones that vary most. That distinction is exactly what a spectroscopic soft sensor needs. A Raman spectrum has 701 wavenumber channels (the 701 points along the spectrum's cm⁻¹ axis, each one a light-intensity reading) that are massively collinear — neighboring channels move together — and only a handful of underlying chemical factors actually drive titer. PLS projects those 701 correlated channels onto a few latent components (the suite selects five by inner cross-validation — repeatedly splitting the training data into a part to fit on and a held-out part to score on, so the choice is judged on data the model did not see) and regresses titer on them, sidestepping the catastrophe an ordinary least-squares fit would suffer on 701 collinear predictors with a few hundred rows, where the normal equations are near-singular and the coefficients blow up.

The mechanism is worth a sentence because it is what makes PLS reviewable. PLS extracts its components one at a time (the classic NIPALS algorithm): it finds the weight vector that maximizes the covariance between a projection of X and y, computes the resulting score, deflates both X and y by what that component explained, and repeats on the residual — so each new component captures predictive structure the earlier ones missed. Two artifacts fall out for free and are exactly what a reviewer reads: the regression coefficient at each wavenumber (how much that channel moves the prediction) and the VIP score (variable importance in projection — roughly, how much each input contributes across all retained components, with a value near or above one marking an influential band). A VIP peak sitting on the protein amide backbone band — the directly causal signal for a titer sensor — is the evidence that the model is right for the right reason (a peak on a glucose band is weaker evidence, since glucose only co-varies with titer as the culture grows); a VIP peak on an empty stretch of baseline is the evidence that it is exploiting an artifact. No deep model hands a reviewer that pair of artifacts.

PLS is roughly forty years old, it is the documented incumbent in commercial spectroscopic PAT (Process Analytical Technology — the discipline of measuring quality in-line, in real time, rather than waiting for an offline lab result), and it is genuinely hard to beat in the small-data regime [1][2]. The whole industry of in-line Raman and NIR soft sensors — glucose, lactate, and titer (the concentration of the product antibody in the culture, in g/L, and the soft sensor's main prediction target) (production), with closed-loop glucose control on the same chemometrics real but still largely (pilot) — runs on PLS chemometrics, with peer-reviewed in-line Raman + PLS models hitting R² above 0.9 for nutrients, metabolites, and IgG titer (IgG being immunoglobulin G, the antibody class most therapeutic mAbs belong to) in 10-litre bioreactors, and the dominant commercial monitoring suites (Sartorius SIMCA, AspenTech ProMV) are productized PCA/PLS with the Hotelling's T² and SPE charts the release chapter dissects [2] (production). When in doubt in bioprocess, you start here — and you very often stay here.

Plain linear regression sits one step below PLS and is worth keeping in view, because the regularized linear models are themselves underrated bioprocess workhorses. Ridge regression (an L2 penalty) tames collinearity by shrinking coefficients smoothly toward zero; LASSO (an L1 penalty) does feature selection by driving some coefficients exactly to zero, which is precisely what you want when a few of many metabolite features actually matter; elastic net blends the two. The release predictor uses an L2-regularized logistic regression for exactly the reasons this family is favored: it is calibrated, its standardized coefficients read directly as log-odds an investigator can interpret (the log of the odds of passing — a negative coefficient pushes a batch toward failure, a positive one toward passing, and a larger magnitude means a stronger push) — in the suite, end_viability_pct (end-of-run cell viability) carries a −2.85 coefficient and peak_lactate_g_L (peak lactate, a waste metabolite) a +2.74, both signed the way process chemistry predicts (poor viability and high lactate-driven stress both raise failure risk) — and it has almost no room to overfit a few hundred rows. The common thread across linear regression, ridge/LASSO, PLS, and PCA is low variance and high transparency — the two properties small-data GMP work prizes above raw flexibility, and the two that the validation half of this chapter turns into a requirement rather than a preference.

Tree ensembles and gradient boosting — tabular, nonlinear, interpretable enough

When the data is tabular (process parameters, metabolite summaries, categorical media or clone identifiers) rather than spectral, and the relationship is nonlinear or full of interactions, the next rung is tree ensembles. A single decision tree recursively splits the feature space into axis-aligned boxes, choosing each split to reduce error most; alone it overfits badly. A random forest averages many trees grown on bootstrapped samples and random feature subsets, which decorrelates them and cuts variance. Gradient boosting (XGBoost, LightGBM) instead grows trees sequentially, each new tree fitting the gradient of the loss left by the ensemble so far — each correcting the last one's residual — which is usually the strongest tabular learner available. Their bioprocess home is the release predictor and the manufacturing-operations models: the suite's viral_lrv.py uses a gradient-boosted regressor for the virus-filtration step's log-reduction value (LRV — how many factors of ten the filter removes the virus by, so an LRV of 4 means a 10,000-fold reduction) and coldchain.py a gradient-boosted lane-risk classifier, each predicting an outcome from a few dozen engineered features. They handle nonlinearity and mixed feature types out of the box, they are robust to feature scaling, and — crucially for review — they emit feature importances and partial-dependence plots, so an investigator can see which feature drove a prediction; the LRV model attributes most of its signal to feed HMW fraction (the proportion of high-molecular-weight aggregates entering the filter) and flux decay (how much the filter's throughput drops as it clogs), both physically sensible drivers of membrane retention (how much the membrane holds back). They are not magic on small data: a boosted ensemble can overfit a few hundred rows as eagerly as a neural net, and they extrapolate flat — a tree cannot predict a value outside the range it saw, so beyond the training envelope it simply returns the edge. But on tabular, moderate-sized bioprocess data they are a sensible, defensible default, interpretable enough to survive review even if not as transparently as a linear coefficient.

Gaussian processes and Bayesian optimization — when each experiment is precious

The third rung is the one built for bioprocess's defining scarcity: experiments cost weeks and a fortune, so you want a model that tells you where to run the next one. A Gaussian Process (GP) is a model that returns, at every point in the input space, not just a predicted mean but a calibrated uncertainty — a full predictive distribution, narrow where you have data and wide where you do not — governed by a kernel (commonly Matérn or squared-exponential) that encodes how similar two settings' outcomes should be as a function of their distance. Formally it treats the unknown function as a draw from a distribution over functions and conditions on the observed points; the math is closed-form but cubic in the number of training points. The honest error bar is itself a governance asset — a model that says "I don't know here" is far easier to trust than one that guesses confidently, and it is the only rung on the ladder that volunteers its own out-of-distribution warning rather than having to be monitored for it.

That uncertainty field is what powers Bayesian Optimization (BO): fit a GP to the experiments run so far, use an acquisition function (Expected Improvement, or its multi-objective cousin Expected Hypervolume Improvement when you must trade titer against quality) to pick the next experiment that best balances exploiting the current best region against exploring uncertain ones, run it, refit, repeat. The result is a feedback loop that reaches a competitive optimum in materially fewer runs than a fixed factorial design-of-experiments grid — the engine behind process development and media optimization, supported across recent peer-reviewed bioprocess work reporting 3–30× fewer experiments than standard DoE (research, peer-reviewed-independent) [3]. The suite makes it concrete: bayesopt_doe.py optimizes a feed policy and reaches a final titer of 6.269 g/L in 15 Bayesian-optimization runs versus 6.246 g/L in a 25-run factorial grid — matching and slightly beating the grid optimum with ten fewer runs. One bioprocess wrinkle the demo glosses, though: unlike the fast in-line titer the suite's bayesopt_doe.py optimizes, the quality objectives a real campaign trades against — aggregate (clumped-together antibody molecules), charge variants (slightly modified product forms that shift the molecule's electrical charge), host-cell protein, all of them undesirable impurities you want to minimize — are measured offline and land days after each run, so real bioprocess BO must optimize against slow, costly, delayed labels, the same lag the unsolved-part section returns to for drift.

Two design choices carry most of a GP's behavior. The kernel is the prior on smoothness: a squared-exponential kernel assumes the response surface is very smooth (nearby settings give nearly identical outcomes), while a Matérn kernel allows rougher surfaces and is the safer default for noisy bioprocess responses; the kernel's length-scale, fit from the data, says how far an observation's influence reaches. The acquisition function is the policy that turns the GP's mean-plus-uncertainty into the next experiment: Expected Improvement weighs the chance and size of beating the current best, the Upper Confidence Bound adds a tunable multiple of the uncertainty to the mean to dial exploration up or down, and Expected Hypervolume Improvement generalizes the idea to several objectives at once — the realistic bioprocess case, where you are never optimizing titer alone but titer against aggregate, charge variants, and host-cell protein together. GP-BO is the rare case where a more sophisticated model is exactly what small data calls for, because its sophistication is spent on quantifying ignorance rather than on raw capacity. Its cost is poor scaling — classical GPs are cubic in the number of training points — which is precisely no obstacle when you have thirty experiments.

Neural networks — MLP, 1D-CNN, autoencoders/VAE, transformers

At the top of the ladder sit the neural networks, in roughly increasing order of appetite for data:

• A multilayer perceptron (MLP) is a stack of fully-connected layers with nonlinear activations — a universal nonlinear regressor that can in principle approximate any function. It can fit anything given enough data; on a few hundred bioprocess rows it mostly fits noise. Its honest bioprocess home is as the residual learner inside a hybrid model, where mechanistic equations carry most of the signal and a small net mops up what the physics missed — the suite's hybrid_model.py uses an 801-parameter network for exactly this, and the hybrid-modeling chapter builds it out.
• A 1D-CNN treats a spectrum as a one-dimensional signal and learns local spectral features with shared convolutional filters, pooling them into a compact representation before a dense head. This is the architecturally correct deep model for Raman or NIR — it respects the translational structure of a spectrum the way a 2D-CNN respects an image — and it is the deep model we benchmark against PLS below. A reviewer's instinct that a CNN "learns its own preprocessing" is right: the convolutions play the role PLS's smoothing and dimensionality reduction play, which is exactly why the CNN's advantage tends to appear only on large multi-instrument datasets, not on one clean batch.
• Autoencoders and variational autoencoders (VAE) are unsupervised: they compress data to a low-dimensional latent code and reconstruct it, learning "normal" structure by minimizing reconstruction error. Their bioprocess use is anomaly detection (a high reconstruction error flags an out-of-family batch, a neural cousin of MSPC's SPE statistic) and, for the VAE — which learns a probabilistic latent space — generating plausible synthetic samples.
• Transformers are the attention-based architecture behind large language models, learning long-range dependencies by letting every position attend to every other. On bioprocess time series they remain largely (research) — a fed-batch run is too short and too few to feed an attention model that hungry; where transformers genuinely earn their place in this domain is on sequence and text data — protein-language models for molecule and clone work, and LLMs over documents in the generative-AI chapter — not on a six-batch fed-batch dataset.

A practitioner's quick map of the ladder onto the question each rung answers: spectra → titer/metabolite (collinear, hundreds of channels) is PLS, every time; unsupervised batch fingerprint / monitoring is PCA with its T² and SPE; tabular outcome from a few dozen engineered features (release pass/fail, harvest load, lane risk) is a gradient-boosted ensemble; which experiment to run next under brutal scarcity is a GP with Bayesian optimization; anomaly against "normal" is an autoencoder or the PCA-SPE statistic; and sequence or free text (clone sequences, deviation narratives) is where the transformer finally earns its keep. Time-series fed-batch trajectories, the case newcomers most expect deep learning to own, are precisely the case where it loses — too few independent batches, too much collinearity, too high a reviewability bar. That ordering matters because it is also, almost exactly, the order in which a model's data appetite grows and its defensibility shrinks. Which brings us to the central argument of the chapter.

Why deep learning rarely beats PLS on bioprocess data

The single most common mistake a newcomer makes in bioprocess ML is to reach for deep learning because it is powerful, and to be surprised when it loses to a linear model from the 1980s. This is not bad luck or bad tuning; it is structural, and it has four interlocking reasons.

The bias-variance trade-off, in plain terms. Every model's expected error decomposes into three pieces: bias² (error from being too simple to capture the truth), variance (error from being so flexible that it fits the noise in this particular training sample), and an irreducible noise floor. A high-capacity model — a deep net with hundreds of thousands of parameters — has low bias but enormous variance: with few training examples, it has the freedom to fit the idiosyncrasies of those examples and generalizes poorly. PLS, with a handful of latent components, has higher bias but far lower variance. On small data, variance dominates the total error, so the lower-variance model wins the only comparison that counts — held-out accuracy (accuracy on rows set aside before fitting and never seen during training, the one honest measure of how a model will do on new data). This is not a knock on deep learning; it is the bias-variance trade-off doing exactly what the textbook says it does, in the regime where bioprocess lives. The corollary is that the lever that most reduces error in this regime is more independent batches, not more layers — and more batches is precisely what bioprocess cannot cheaply buy.

Sample efficiency, and how little data there really is. Recall the cold-start reality: the binding constraint is not the number of rows but the number of independent batches, which grows by ones, slowly, at the cost of weeks each. A spectrum gives you 701 columns but a batch gives you one genuinely independent observation, so a table with hundreds of thousands of cells (rows times columns of spreadsheet entries, not biological cells) can still carry the statistical weight of six points. A deep network's parameter count routinely exceeds its independent-example count by orders of magnitude. PLS's effective complexity is a few latent components. Deep learning's spectacular wins came from datasets of millions of examples (ImageNet, web-scale text); bioprocess offers six batches. There is no architecture trick that manufactures information the data never contained — a lesson the hybrid-modeling chapter turns into a positive strategy by injecting mechanistic knowledge the data never had to provide.

Extrapolation, where manufacturing actually operates. A validated process runs in a tight, characterized window; the moments you most need a model are excursions and edges — exactly where there is no training data. A linear model extrapolates predictably (you can see where it is heading and bound the error), a tree ensemble extrapolates flat (it pins to the edge of what it saw), and a GP tells you honestly that its uncertainty has exploded. A deep net extrapolates unpredictably and confidently — it returns a crisp number with no signal that it has wandered off the map, because nothing in its training penalizes overconfidence outside the data. In a domain where a confidently-wrong prediction can mis-feed a culture or wave through an off-spec lot, predictable extrapolation is worth more than raw in-distribution accuracy.

Interpretability for review. This reason has no analogue in consumer ML and dominates in pharma. A PLS model exposes its regression coefficients and variable-importance-in-projection (VIP) scores; an investigator can point at the wavenumbers driving a prediction and check them against known chemistry — a glucose band, a protein amide signature — and confirm the model is right for the right reason rather than exploiting a spurious correlation (a probe artifact, a batch-confounded baseline). A deep net is, to a reviewer, a black box — and a black box is a hard thing to defend in front of a regulator who must understand why a model decided what it did about a medicine. Post-hoc explainability tools (SHAP values, saliency maps) help, but they are approximations of an opaque model's behavior, not the model's actual reasoning, and a reviewer can reasonably distrust an explanation that the model itself did not produce and that can change with the explainer's settings. The hybrid models chapter makes the same point from the other side: a model whose prediction can be decomposed into "what the physics said" and "what the data added" slots into validation far more comfortably than an opaque one. Interpretability is not a nicety here; it is a gating requirement for GMP use — and it is the reason the choice of model and its validatability cannot be separated.

The honest corrective the field repeatedly needs: deep learning is not generally superior on small bioprocess data. The most-cited apparent counterexample — the Boehringer Ingelheim work predicting 16 quality attributes in-line and in near-real-time during Protein A capture — is genuinely excellent and genuinely (pilot), but its best overall model was k-nearest-neighbors, a classical distance-based method with no neural network in it, selected over PLS and PCR because it lowered the high-molecular-weight error about three-fold [1]. It is frequently miscited as proof of a "deep-learning Raman wave"; it is, if anything, the strongest single piece of evidence for classical chemometrics on real spectra. Where head-to-head deep-versus-PLS Raman benchmarks exist, deep learning typically matches rather than decisively beats PLS, and its advantage materializes mainly on large multi-instrument datasets, at a large cost in data, compute, and reviewability [1][4] (pilot/research). The practitioner's rule that falls out of all four reasons is blunt: start at the bottom of the ladder and climb only when the data forces you to — which, in bioprocess, is rarely.

The model ladder mapped onto data appetite: the methods that do real production bioprocess work — PLS/PCA, tree ensembles, and GP-BO — sit inside the small-data band where bioprocess actually lives, while MLPs, CNNs, and transformers sit to its right, needing far more data than six batches can supply; climb the ladder only when the data forces you to. Original diagram by the authors, created with AI assistance.

The head-to-head, in code

The example suite makes this argument runnable rather than rhetorical. Two modules fit the same titer-from-Raman problem on the same golden-batch spectra: soft_sensor_pls.py (a PLS whose five components are chosen by inner cross-validation) and soft_sensor_deep.py (a compact 1D-CNN). The CNN treats each 701-channel spectrum as a one-dimensional signal, runs two convolutional blocks over wavenumber (an 8-filter kernel of width 15, then a 16-filter kernel of width 11, with pooling), and regresses titer through a small dense head — the architecturally correct deep model for a spectrum. Crucially, both modules pin the same preprocessing as part of the model: a StandardScaler fit on the training split only and carried with the weights, so neither leaks. The point of running them side by side is not to crown a winner but to show, on real numbers, that orders of magnitude more parameters buy no better held-out result on small, clean spectra.

# soft_sensor_deep.py — a compact 1D-CNN over the 701-wavenumber Raman signal,
# benchmarked head-to-head against the PLS baseline in soft_sensor_pls.py.
class SpectraCNN(nn.Module):
    """Two conv blocks over wavenumber, then a small dense head."""
    def __init__(self, n_wavenumbers: int):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv1d(1, 8, kernel_size=15, padding=7), nn.ReLU(),
            nn.MaxPool1d(4),
            nn.Conv1d(8, 16, kernel_size=11, padding=5), nn.ReLU(),
            nn.AdaptiveAvgPool1d(8),
        )
        self.head = nn.Sequential(
            nn.Flatten(), nn.Linear(16 * 8, 32), nn.ReLU(),
            nn.Dropout(0.2), nn.Linear(32, 1),
        )

    def forward(self, x):              # x: (batch, 1, n_wavenumbers)
        return self.head(self.features(x))


if __name__ == "__main__":
    from soft_sensor_pls import train_pls
    pls = train_pls()                  # the 40-year-old chemometric baseline
    cnn = train_cnn()                  # thousands of parameters of deep model
    print(f"  PLS    : R2={pls['r2']}  ({pls['n_params']} coefficients)")
    print(f"  1D-CNN : R2={cnn['r2']}  ({cnn['n_params']} parameters)")
    print(f"  PLS uses {cnn['n_params'] / pls['n_params']:.0f}x fewer params "
          f"and is not beaten on R2.")

Running the head-to-head prints the lesson directly (numbers from the simulated golden-batch spectra, 235 train / 101 test wavenumber-rows; R² is the 0-to-1 goodness-of-fit, where higher is better, and RMSE — root-mean-square error — is the typical prediction error in the target's own units, g/L of titer here, where lower is better):

Head-to-head: titer from 701-wavenumber Raman, golden batch BATCH-2026-001
  PLS    : R2=0.9944  RMSE=0.127 g/L  (702 coefficients)
  1D-CNN : R2=0.9924  RMSE=0.1488 g/L  (5713 parameters)
  PLS uses 8x fewer params and is not beaten on R2.

Read those two lines carefully, because they are the whole argument compressed. The CNN is not broken — it is a perfectly reasonable model, and on a clean simulated signal it scores R² 0.9924. But PLS, now with real SNV plus Savitzky-Golay preprocessing (SNV — standard normal variate — rescales each spectrum to cancel scatter differences, and Savitzky-Golay is a smoothing-and-derivative filter that sharpens peaks; together they are standard spectral clean-up) and its component count chosen by inner cross-validation under the one-standard-error rule (a convention that picks the simplest model within one standard error of the best, which lands it at five), scores R² 0.9944. So the deep net does not even tie — the 40-year-old linear method slightly edges it, at roughly one-eighth the parameters (702 against 5,713). With an extra dependency on PyTorch, far more compute and a 300-epoch training loop, and a model no reviewer can read, the deep net fails to beat a linear method from the 1980s on the metric that matters — and here it does not even match it. A tie was already a loss for the more expensive, less interpretable model; an outright edge to PLS drives the point home all the harder. On real Raman, with its scatter, probe fouling, and run-to-run drift, the deep model's higher variance usually makes the gap worse, not better, while PLS's reviewable coefficients keep it defensible. Note one honest caveat the data chapter insists on: this demonstration splits wavenumber-rows within a single batch to compare architectures cleanly; a deployable soft sensor must be validated under the batch-grouped split, which is the harness that makes any of these numbers admissible. The architecture comparison is fair only because both models see the identical in-batch split.

Nested cross-validation, so the score you report is the score you would get. There is a subtle way to flatter any model on scarce data, and it is the most common one. If you choose a hyperparameter — the number of PLS components, a regularization strength C — by cross-validation, and then report that same cross-validation score, you have tuned and evaluated on the same batches: the score is optimistic, because the selection already peeked at every fold. Nested cross-validation removes the bias by putting the tuning inside an inner loop run only on the training portion of each outer fold, and reporting only the outer fold's score — the fold that was never used to pick anything, so its number is honest. The suite now demonstrates this in release_predict.py, which tunes the logistic regression's C in an inner loop and reports the outer loop's verdict:

logistic release predictor (NESTED 5x5 CV; C tuned in the inner loop):
  AUROC = 0.923 (95% CI (0.781, 1.0))   AUPRC = 0.805 (95% CI (0.521, 1.0))
  naive (tuned-and-read-off the same folds) AUROC = 0.968  -> optimism removed by nesting = +0.045  (selected C=100.0)
  calibration of the probabilities: Brier = 0.0524  ECE = 0.0769  (lower is better-calibrated)

AUROC and AUPRC are both 0-to-1 scores of how well the model ranks passing-versus-failing batches (higher is better; AUPRC is the more honest of the two when failures are rare), and each 95% CI (confidence interval) is the plausible range around that score. The naive AUROC of 0.968 is what you would report if you tuned and read off the same folds; the honest, nested number is 0.923 — the nesting strips +0.045 of optimism that the naive estimate would have quietly claimed as accuracy. On a 120-batch cohort with only ten OOS positives that gap is the difference between a number a regulator can trust and a number that flatters itself, and it is exactly the kind of self-deception the validation half of this chapter exists to forbid. The same nested run also measures whether the predicted probabilities are honest, not just whether the ranking is — the suite now reports the Brier score (0.0524) and the Expected Calibration Error (ECE) (0.0769) on the nested-CV predictions, and batch_outcome.py applies an isotonic recalibration on top, so "calibrated confidence" is a measured number rather than a claim.

Uncertainty quantification: a number plus its honest spread

A point prediction is a confident lie. "Titer is 6.3 g/L" tells a reviewer nothing about whether the true value is 6.2 or 4.0, and in a regulated setting that missing spread is the whole decision. A quality unit does not act on a slide that reports an accuracy; it acts on a number with an interval ("the log-reduction value is at least 4.1") or a calibrated probability ("this lot passes with probability 0.93"). Uncertainty quantification is the discipline of attaching that spread, and the book has been invoking it piecemeal — the viral-safety quantile bands, the release bootstrap interval and calibration, the hybrid twins — so it earns one clean treatment here. Three practical families do the work, in rising order of how much they assume.

Quantile regression fits the edges of the distribution directly. Instead of training one model on the mean (squared-error loss), you train extra models on the pinball (quantile) loss, which asymmetrically penalizes being above versus below a target percentile, so a model fit at the 5th and 95th percentiles brackets a 90% band straight out of the predictor. The suite's viral_lrv.py does exactly this: alongside its point gradient-boosted LRV it fits two more quantile-loss gradient-boosted regressors at the low and high percentiles, so every predicted log-reduction value arrives as a band rather than a bare number — and the module checks the band's empirical coverage on held-out runs, because an interval that does not actually catch what it claims to is no better than the point it dresses up.

Conformal prediction turns any model's errors into an interval with a distribution-free, finite-sample coverage guarantee — formalized by Vovk and colleagues and brought to a wide audience by Angelopoulos and Bates' tutorial. The intuition is disarmingly simple: run the model on a held-out calibration set, rank how wrong it was on each, and use a chosen quantile of those past errors as the half-width of every future interval. If the calibration errors are representative of the future ones, the interval covers the truth at the stated rate — no Gaussian assumption, no asymptotics, any underlying model. The one thing it must assume is exchangeability: that the calibration points and the new point are drawn from one indistinguishable pool, order irrelevant.

Ensembles and the bootstrap estimate spread by resampling. The percentile bootstrap that dataio.bootstrap_ci already computes for the release predictor's AUROC and AUPRC resamples the held-out predictions with replacement, recomputes the metric a thousand times, and reads the interval straight off the percentiles of that spread — so the reported metric arrives as a range a regulator can read, not a lone point that hides how few batches it rests on. A companion question is whether predicted probabilities are honest, which calibration measures: the Brier score and Expected Calibration Error (ECE) that batch_outcome.py and release_predict.py report ask whether events the model calls "90% likely" actually happen about 90% of the time, and an isotonic recalibration repairs them when they do not.

The caveat that ties to the book's thesis. Every one of these guarantees rests on exchangeability — that tomorrow's batch looks statistically like the held-out batches the interval was built from. That assumption is exactly what bioprocess breaks. Under distribution shift, an out-of-distribution input, or a generatively-designed candidate far from the training set, the calibration set no longer represents the new point, and the interval silently loses its coverage: a held-out 90% band on a drifted batch can be confidently, narrowly wrong. This is the small-data-ceiling, drifting-process regime the data chapter named, viewed from the uncertainty side — an interval is only as honest as the exchangeability behind it, which is why the applicability-domain gate that flags an out-of-envelope input is the necessary partner to any interval, and why the unsolved-part section below treats drift as the standing threat it is. The GMP payoff is direct: a release or advisory decision needs the interval or the calibrated probability, not the point — the viral-safety LRV is reported as a band and the release verdict as a calibrated probability for precisely this reason — but the apparatus must report when its own assumption no longer holds rather than handing over a crisp, comfortable, wrong number.

Anatomy of a validated PLS soft-sensor package

A model that earns a place in a GMP plant is not a .pkl file with a good R² — it is a validated package, and what travels alongside the fitted coefficients is what makes it usable for a decision about a medicine. Dissect the package the way a quality reviewer would, and the whole second half of this chapter is laid out as fields.

One validated soft-sensor package, fully unpacked: the build provenance that pins it to an exact dataset hash, scaler, and preprocessing; the green validation core with acceptance criteria, operating range, and advisory scope; the interpretability artifacts (PLS coefficients and VIP scores) that a deep net cannot supply; the locked-model and PCCP lifecycle; and the GAMP 5 / CSA / FDA-credibility governance that makes it a validated object rather than a model file. Original diagram by the authors, created with AI assistance.

Read the card top to bottom and the validation paradigm is concrete. The build block is provenance: the feature contract (wn_400…wn_1800, named so a renamed or reordered input fails loudly rather than silently mis-predicting), the SNV plus Savitzky-Golay preprocessing pinned as part of the model (preprocessing is the model — standard normal variate scatter correction and the Savitzky-Golay derivative must be fit on training data only, or every prediction leaks the test set's statistics back into itself), the fitted scaler that must travel with the coefficients, the five latent components selected by inner cross-validation, and the training dataset pinned by its sha256 so "which data trained this?" is never a guess but a hash that either matches or does not. The green core is what validation produced, not what the model hoped for: held-out R² and RMSE against written acceptance criteria set before the run (the suite's gate is R² > 0.85; the PLS sensor clears it at 0.9944), the qualified operating range outside which the prediction is not valid, and the intended-use scope — advisory, human decides, never autonomous CQA control. That scope line is not a disclaimer; it is the single field that sets how heavy the rest of the validation must be. The amber block is the advantage that justified choosing PLS in the first place: per-wavenumber regression coefficients and VIP scores a reviewer can read against known chemistry, the artifact a deep net cannot supply. These reviewable artifacts are now runnable rather than asserted — soft_sensor_pls.py actually computes the VIP scores (its top bands fall at roughly 1270–1276 cm⁻¹, the protein Amide III backbone region — the directly causal signal for a titer sensor, since titer is protein concentration, exactly the band a reviewer wants to see driving the prediction rather than a merely co-varying nutrient band) and a per-prediction applicability-domain gate (a Hotelling T² and SPE check that flags any spectrum lying outside the model's training envelope before its number is trusted). The rose lifecycle block holds the locked status, the PCCP that governs future change, and the next revalidation date. The violet governance panel carries the GAMP 5 category, the CSA risk assessment, the FDA credibility tier, the four-eyes signatures, and the lineage edges — trainedOn the pinned dataset, validatedUnder the PCCP, monitoredBy the drift detectors, supersedes v1 — so the package is a node in an auditable graph, not an orphan file, the kind of instance graph Book 4 builds and surveys in production. A model file has weights; a validated package has all of this — which is the only reason it is allowed near a batch.

The ontology underneath: the same graph that grounds the model also guards its inputs

Three of the package's hardest problems — which data trained this, are those data complete and in range, and how do I split them honestly — turn out to be the same problems an ontology was already built to solve. A knowledge graph is not decoration bolted onto a model; in bioprocess it is what makes the model both FAIR (findable, accessible, interoperable, reusable) and trustworthy, and it earns one short treatment here because the validation half of this chapter rests on it.

The lineage edge is the grouping key. The data chapter insisted that a deployable soft sensor be validated under a batch-grouped split, never a random one — and the group it must split on is exactly the lineage edge Book 4 models as bp:derivedFrom, the one transitive spine that roots every drug-substance lot, hop by hop, in the frozen working cell bank WCB-CHO-001. Leave-one-batch-out cross-validation is bp:derivedFrom read as a grouping key: every row sharing a lineage back to the same fed-batch run is one group, held out together, so a fold can never train on one wavenumber-row of a batch and test on another. The suite encodes this as the batch_id group the dataio layer keeps on every returned frame; the ontology is where that same fact is typed — a measurement is a BFO continuant or occurrent distinct from the run it belongs to, so the graph cannot quietly confuse a spectrum with the batch that produced it the way a flat column name can. The grouping key is not a convention you must remember; it is an edge in the graph.

The SHACL release shape is also the training-data input contract. The feature contract wn_400…wn_1800 that fails loudly on a renamed column is the brittle, string-based cousin of a far stronger guarantee. Book 4's release gate is a set of SHACL shapes — bp:ReleaseShape requires every released lot to carry exactly one in-range value for every required CQA (sh:minCount 1, sh:maxCount 1, sh:minInclusive 95.0 on monomer purity, and so on), validating in a closed world where a missing required result is a failure now, not an open "unknown" the way OWL would treat it. That same shape is exactly the check a training set needs: complete (no batch silently missing an offline assay), single-valued (no duplicate HCP reading sneaking two rows in for one lot), and in declared range (no impossible negative titer from a parser bug). Validating the training table against the release SHACL shape before a fit is the data-quality gate that MANIFEST.sha256 only pins — the hash proves which bytes trained the model; the SHACL shape proves those bytes are complete and admissible. One shape guards both the lot at release and the model at training.

Semantically-grounded features, and the graph as the LLM's ground truth. A feature pulled by its ontology IRI — the monomer purity bp:DS-001 carries as bp:monomerPct, typed and unit-bearing — survives a column rename that would silently break a fragile df["monomer"], because the IRI means the same thing across the ELN, LIMS, and MES the data shadow is scattered across. The same graph is what a GraphRAG assistant must be grounded against: when a model or a copilot answers "what was this lot derived from?", it answers by walking the typed bp:derivedFrom edges and citing them rather than inventing a plausible chain — the ontology is what keeps a fluent model honest, which is the same discipline this chapter demands of every prediction. The validated package's lineage edges (trainedOn, validatedUnder, monitoredBy, supersedes) are not loose labels for the same reason: they are typed relations in that graph, queryable the way bp:derivedFrom is, so "which model scored this batch, trained on which pinned data, under which PCCP?" is a path through the graph, not a hunt through filenames.

The GMP validation paradigm

The second half of "which model" is "how do you prove you can trust it." In consumer ML the answer is a held-out test score. Under GxP it is a far larger apparatus, and understanding its shape is as important as understanding the models.

Locked versus continuously-learning models. A GMP (Good Manufacturing Practice — the regulations governing how a medicine is actually made) process must be validated — proven, frozen, and held under change control so that what you make tomorrow is what you proved yesterday. A model that keeps learning is, by definition, a system that keeps changing, which is the thing validation forbids without re-qualification: yesterday's signed evidence describes a model that no longer exists. The resolution the industry and regulators have converged on is the locked model: weights, preprocessing, scaler, and operating range all version-pinned and unchangeable in place. It does not learn on the fly; it is loaded in eval() mode and never updated by the data it scores. The MLOps chapter builds the full lifecycle around this; here it is enough to state the rule that governs every model in this book — locked-then-relearn, never continuously-learning, for anything touching a critical quality attribute.

The Predetermined Change Control Plan (PCCP). A locked model that can never change would be a dead end, because the world moves and the model drifts. The PCCP is the mechanism that lets a model change on purpose without a fresh regulatory negotiation each time: a pre-approved, written specification of how the model may be retrained — which data sources, which fixed algorithm and hyperparameters (the architecture is frozen even when the weights may refresh), what acceptance criteria the new version must clear, and the rollback plan if it does not. A retrain that stays inside the PCCP's envelope is a planned, documented event rather than an unforeseen change; a change outside the envelope (a new architecture, a new input) falls out of the plan and back into full re-qualification. The PCCP is the bridge across the validation-versus-learning gap, and the MLOps chapter shows it driving a real retraining loop.

GAMP 5 and Computer Software Assurance (CSA). Pharma already had a discipline for trusting software before ML arrived: GAMP 5 (Good Automated Manufacturing Practice), the risk-based framework for validating computerized systems, which categorizes software by how custom and how critical it is and scales the validation effort to match. Computer Software Assurance (CSA) is the FDA's 2022-era reframing of that effort — a deliberate shift away from exhaustive, document-everything testing toward critical thinking and risk-based assurance, spending validation effort where the patient risk is rather than uniformly across every screen and field. Applied to ML, GAMP 5 and CSA say: a soft sensor that only advises a human carries less risk — and a lighter assurance burden — than one wired to act on a CQA, and the evidence you gather should be proportional to that risk. The ISPE GAMP AI guide (first edition, July 2025) extends this established computerized-system-validation thinking specifically toward AI/ML — risk-based, lifecycle-oriented, demanding ongoing performance evidence rather than a one-time test — translating "validate your software" into "validate your model and monitor it forever" [5].

The FDA 7-step model-credibility framework. The FDA's thinking on the credibility of computational models — anchored in the ASME V&V 40 consensus standard and echoed in the agency's 2025 draft guidance on AI for regulatory decision-making — gives the risk-proportionate spine. In outline, the steps are: state the question of interest; define the model's context of use (exactly what role it plays in the decision); assess the model risk — the product of how much the decision relies on the model (model influence) and how severe being wrong would be (decision consequence); plan the credibility activities proportional to that risk; execute them; document the results; and decide on adequacy for the stated use, re-assessing whenever the use changes. (The agency's own write-ups enumerate a few more sub-steps and eight categories of credibility evidence; this chapter uses the "7-step" shorthand for the same risk-informed framework — the count differs, the logic does not.) The deep idea is simple and powerful: a model is not trustworthy because it ran; it is trustworthy because evidence was produced and checked against a pre-stated acceptance criterion, at a rigor matched to the consequence of being wrong [6]. A soft sensor that merely advises sits low on the risk axis and needs lighter evidence; a model wired to release a lot sits at the top and needs the heaviest.

The example suite's run_all.py harness is a small software analogue of exactly this — the Computer Software Assurance view of the whole suite at once. It is deliberately not another model. It runs every module as a subprocess, treats each module's terminating assert as a pre-stated acceptance gate, captures the exit code as the verdict (a clean exit is evidence the gate cleared; a non-zero exit is documented evidence that the model fails its own criterion and, under a PCCP, blocks deployment rather than being quietly ignored), and records the SHA-256 of every dataset each model was fitted on — the "frozen at validation" pin that turns a run into a record. The output reads as a validation summary, not a benchmark:

# run_all.py — a model is credible because EVIDENCE cleared a pre-stated gate
# on a PINNED dataset, not because it ran. The script analogue of the FDA framework.
@dataclass(frozen=True)              # shown abridged — the real struct also
class ModelEvidence:                 # carries a `chapter` int and a `slow` flag
    module: str            # the model that produces the evidence
    gate: str              # the pre-stated acceptance criterion (its own assert)
    datasets: tuple        # the data it was fitted on, pinned by sha256
    passed: bool | None    # did the evidence clear the gate?

LEDGER = [
    ModelEvidence("soft_sensor_pls.py",  "Raman->titer R2 > 0.85 (within-batch interpolation)",
                  ("raman_spectra.parquet",)),
    ModelEvidence("mspc.py",             "MSPC flags ONLY the OOS batch; SPE points at HCP",
                  ("hplc_results.csv",)),
    # ... every module ends in an assert == an acceptance criterion in a protocol.
]

Run it across the suite and it emits a machine-checkable credibility ledger — module, chapter, acceptance gate, the dataset hash it was fitted on, and a pass/fail — closing with the line that matters most:

model-credibility evidence harness — Book 5 suite
  acceptance gate per model is the module's own assert

  PASS  soft_sensor_pls.py       ch11  [raman_spectra.parquet:4d7f12c463eb]
        gate: Raman->titer R2 > 0.85 (within-batch interpolation)
  PASS  soft_sensor_deep.py      ch11  [raman_spectra.parquet:4d7f12c463eb]
        gate: deep soft sensor R2 > 0.85
  PASS  mspc.py                  ch18  [hplc_results.csv:e3d529ac583b, batches.csv:e3a78c7291c8]
        gate: MSPC flags ONLY the OOS batch; SPE points at HCP
  PASS  batch_mvda.py            ch18  [(simulator):(simulator)]
        gate: DTW+unfold+MPCA flags the stressed batches as trajectory outliers
  PASS  lifecycle_retrain.py     ch22  [fedbatch_state.parquet:aba381af160e, offline_assays.csv:d5cae737bcad]
        gate: challenger beats champion, back in control -> governed PROMOTE
  ... (21 models total)

credibility summary: 21/21 models cleared their acceptance gate on the pinned datasets
  NOTE: passing the gate is necessary, not sufficient — GMP credibility
  also needs intended-use scope, change control, and human oversight.

That closing NOTE is the discipline: the harness proves the form of credibility — a stated gate, evidence against it, a pass/fail, a data hash — but it cannot prove the substance, which still needs documented intended use, change control, and a human in the loop. Twenty green gates on pinned data is the floor, not the ceiling. The two soft-sensor rows sharing the identical raman_spectra.parquet:4d7f12c463eb hash is the head-to-head's reproducibility made auditable: same fuel, same gate, two engines.

Draft Annex 22, at altitude. The draft EU/PIC/S GMP Annex 22 (released for consultation July 2025) draws the hardest line of all: it applies only to static models — parameters fixed, deterministic output — and requires that AI used in critical GMP applications be static (locked), explicitly forbidding a model that adapts its performance during use and excluding self-learning, generative AI, and LLMs from those critical uses, while demanding a predetermined change-control approach for any update [7]. In other words, the draft codifies locked model plus PCCP as the only acceptable pattern where a model touches product quality, patient safety, or data integrity — the continuously-learning model is, for now, regulatorily off the table for critical decisions, and the generative copilots of the GenAI chapter are confined to advisory roles by the same line. (The regulation chapter goes deeper on Annex 22 and the FDA framework; this chapter establishes only the shape every model in the book must fit.)

Model monitoring, explainability, and documentation. Three expectations run through all of the above and recur in every later chapter. Monitoring: a validated model is watched for drift after deployment, because its performance can decay silently as the living process and its hardware move — the MLOps chapter builds the detectors (residual control charts, population-stability indices) the suite's drift.py prototypes. Explainability: a model used for a GMP decision must be interpretable enough that a human can understand and defend its reasoning — the single largest practical reason PLS's reviewable coefficients beat a deep net's opacity in this domain, and a soft requirement that the draft Annex 22 hardens into an expectation. Documentation: intended use, training-data provenance, validation evidence, acceptance criteria, operating range, and change-control history are not paperwork about the model; under GxP they are the model, the difference between a .pkl file and a validated object. The four-eyes-signed, timestamped, hash-pinned record is what it is because regulators read it against the ALCOA+ data-integrity expectations (data that is attributable, legible, contemporaneous, original, and accurate, plus the "+" extensions) and the 21 CFR Part 11 / EU Annex 11 controls on electronic records and signatures — the frameworks that make the package an auditable record rather than just a file.

Tech transfer, multi-site deployment, and equipment re-qualification. A validated model rarely lives at one bench. The same way a process is tech-transferred — qualified at a receiving site against the originating site's evidence before it makes commercial product — a model must be re-qualified when it moves: a soft sensor validated on BR101's Raman probe at the development site is not automatically valid on a different probe, a different bioreactor, or a contract manufacturer's line, because the spectral baseline, the scatter, and the instrument's own drift are part of what the model learned. The practical rule is that the model's intended use and operating range travel with it, but its acceptance evidence must be re-generated at the new site or on the new hardware — a held-out qualification run that clears the same pre-stated gate — and that this re-qualification is a planned event under the PCCP, not an informal copy. Two equipment events sit at the sharp edge of this: a probe swap or re-standardization shifts the spectral baseline enough to be a change-control and requalification trigger rather than a monitoring blip, and a raw-material or column re-lot can move the process into a corner the model never saw. The MLOps chapter builds the governed retraining loop that absorbs these; here the point is that "validated" is always validated for a stated site, instrument, and range, and crossing any of those boundaries reopens the evidence question.

The environment is part of the lock. Reproducibility under GMP means a frozen model produces the same number tomorrow, which is impossible if the software underneath shifts. The companion suite leans on the open-source stack — scikit-learn and PyTorch, permissively licensed (Apache-2.0 / BSD), so the analytics are inspectable and runnable rather than a vendor black box — but an open stack only reproduces if it is pinned: a fixed random seed for every split and fit, a pinned dependency set, and the MANIFEST.sha256 dataset hashes the run_all.py harness records. A locked model whose library versions float is not actually locked, which is why the lock spans weights, preprocessing, scaler, operating range, and the environment that runs them — and why the open-source analytics chapter and the MLOps chapter treat the pinned environment, not just the pinned weights, as the unit of validation.

The unsolved part: a static model watching a moving process

The tension this chapter sets up but does not resolve is the one the whole back half of the book wrestles with: we have just argued that a credible GMP model must be locked, and the data chapter argued that the process it watches is alive and never identical twice. A frozen model aimed at a moving target is, by construction, slowly going wrong — its accuracy decays the moment the cell line adapts, a probe fouls or is swapped, or a new raw-material lot arrives, none of which it is allowed to learn from in place. (A probe swap is a step beyond ordinary drift: replacing or re-standardizing the physical Raman instrument shifts the spectral baseline, so it is a change-control and requalification event under the PCCP, not merely a monitoring signal.) Validation freezes the model precisely when the process refuses to hold still, and Annex 22 makes that freeze a requirement rather than a choice. There is no clean dissolution of that paradox; there is only management of it — lock the model, monitor it for drift, retrain off-line into a new validated version under a PCCP, and promote through a human gate. And even that management has an honest limit: many of the most important signals (a release HCP, an SEC monomer) arrive days after the batch the model scored, so drift is often confirmed only once the slow reference data lands, long after the prediction was acted on. That management loop, the detectors that make it run, and the limits of detecting drift before the reference data arrives are the subject of the MLOps and lifecycle chapter. For now, the lesson is that choosing the model and keeping it true are two halves of one problem, and the validation paradigm exists to hold them together rather than to pretend the tension away.

What this chapter adds to the model suite

This chapter does not contribute a new model so much as frame the suite the data chapter bootstrapped and the later chapters fill in. Two existing artifacts anchor its argument:

• soft_sensor_pls.py and soft_sensor_deep.py — the PLS-versus-1D-CNN head-to-head on the golden-batch Raman spectra — same data (the identical raman_spectra.parquet hash), same target, same in-batch split, two rungs of the ladder — in which the deep model fails to beat the 40-year-old linear baseline on the held-out number (the chapter's central claim made runnable, with the numbers in The head-to-head above).
• run_all.py — the model-credibility evidence harness, and the chapter's CSA contribution. It is deliberately not another model; it runs every module in the suite, treats each one's terminating assert as a pre-stated acceptance gate, records the verdict and the SHA-256 of the data each was fitted on, and emits a 21/21 models cleared their acceptance gate on the pinned datasets ledger — a software-assurance analogue of the FDA 7-step framework, demonstrating in code that credibility is evidence against a fixed criterion on pinned data, not a good run. Every later chapter's module ends in an assert precisely so this harness can read it as an acceptance criterion.

Together they make the chapter's two halves concrete: which engine fits (PLS, on the evidence), and what it takes for that engine to be trusted (a fixed gate cleared on pinned data, with intended use and oversight on top).

Why it matters

The fastest way to waste a bioprocess ML project is to spend it on the wrong half of the problem. Newcomers pour effort into model architecture — deeper nets, fancier optimizers, the latest transformer — and discover that on six batches the elaborate model loses to PLS, costs more, needs a GPU, and cannot be explained to a reviewer. Meanwhile the half that actually decides whether the model ever reaches a plant — validation, locking, documentation, the credibility evidence a regulator demands — gets treated as an afterthought, and the model dies in review. This chapter inverts both instincts. Climb no higher up the model ladder than the data forces you to, because in bioprocess the simplest engine that fits is almost always the one that ships; and treat validation not as paperwork bolted on at the end but as a first-class design constraint that shapes the model choice from the start — which is exactly why interpretability pushes you toward PLS before you have run a single fit. Get those two right and the model is both good and trustworthy; get either wrong and a high R² is worth nothing.

In the real world

The production reality matches the argument with unusual cleanliness. The strongest deployed bioprocess ML is overwhelmingly classical: in-line Raman/NIR soft sensing on PLS chemometrics for glucose, lactate, and titer (production); closed-loop glucose control on the same chemometrics is real but still largely (pilot); and PCA/PLS multivariate monitoring — Sartorius SIMCA, AspenTech ProMV — for continued process verification and golden-batch monitoring (production) [2]. GP-BO is real and growing in process development, with peer-reviewed reports of 3–30× fewer experiments than fixed DoE for media optimization (research/pilot, peer-reviewed-independent) [3]. Deep learning's genuine production foothold is narrow and specific — automated visual inspection of vials, a computer-vision problem with abundant images, which the fill-finish chapter covers — not soft sensing on small spectral or kinetic datasets, where it repeatedly fails to beat the chemometric baseline. The most-cited apparent deep-Raman success was, on inspection, KNN [1]. And the validation paradigm is not aspirational: the ISPE Pharma 4.0 surveys consistently find ML clustered in monitoring and advisory roles, almost never in autonomous control, precisely because the locked-model-plus-PCCP lifecycle and the credibility evidence are hard, expensive, and not something you can pip install. The honest verdict the rest of the book elaborates: the engine is usually simple, and the trust is always expensive.

Key terms

• GxP / GMP — GxP is the umbrella of Good-x-Practice regulations (Good Manufacturing, Laboratory, and Clinical Practice) a medicine is made under; GMP (Good Manufacturing Practice) is the manufacturing member of that family, governing how a batch is actually produced.
• Titer — the concentration of product (the antibody) in the culture, in g/L; the soft sensor's main prediction target.
• Raman spectrum — a light-scattering chemical fingerprint, here 701 wavenumber channels (the cm⁻¹ x-axis of the curve), read in-line to infer concentrations.
• OOS (out of specification) — a result outside the written acceptance limit (e.g. host-cell protein 128 ng/mg against a 100 max), which triggers a formal investigation.
• PAT (Process Analytical Technology) — the discipline of measuring quality in-line and in real time, rather than waiting for an offline lab result; the in-line-measurement role soft sensors serve.
• Cross-validation — repeatedly splitting the training data into a part to fit on and a held-out part to score on, so a model or hyperparameter choice is judged on data the model did not see; nested CV puts the tuning in an inner loop and reports only the untouched outer-fold score.
• R² / RMSE — R² is a 0-to-1 goodness-of-fit (higher is better, 1 is perfect); RMSE (root-mean-square error) is the typical prediction error in the target's own units (g/L of titer), where lower is better.
• AUROC / AUPRC — 0-to-1 scores of how well a classifier ranks pass-versus-fail batches (higher is better); AUPRC is the more honest of the two when failures are rare.
• PLS (Partial Least Squares) — the supervised latent-variable method that projects many collinear predictors (e.g. 701 Raman channels) onto a few components of maximal covariance with the target; the small-data workhorse of bioprocess soft sensing.
• PCA (Principal Component Analysis) — the unsupervised cousin that finds directions of greatest variance by eigen-decomposing the covariance; the engine of multivariate monitoring (MSPC), with Hotelling's T² and SPE statistics.
• Tree ensembles / gradient boosting — random forests (bagged, decorrelated trees) and boosted trees (sequential residual-fitting); the strong, reasonably interpretable default for tabular, nonlinear, moderate-sized bioprocess data, but flat extrapolators.
• Gaussian Process (GP) — a model that returns a calibrated predictive distribution (mean and uncertainty) at every input via a kernel; the surrogate that powers Bayesian optimization; cubic in training points.
• Bayesian Optimization (BO) — the feedback loop that uses a GP's uncertainty and an acquisition function to choose the next experiment, reaching an optimum in far fewer runs than a fixed DoE grid; built for bioprocess's experiment scarcity.
• 1D-CNN — a convolutional network that treats a spectrum as a one-dimensional signal with shared filters; the architecturally correct deep model for Raman/NIR, and the one that still fails to beat PLS on small data (here it scores R² 0.9924 to PLS's 0.9944) at roughly 8× the parameters.
• Calibration (Brier / ECE) — measures of whether a classifier's predicted probabilities match observed frequencies, not just whether its ranking is good: the Brier score (mean squared error of the probability) and the Expected Calibration Error (the gap between confidence and accuracy across probability bins); both lower-is-better, both now reported on the suite's nested-CV release predictions and improved by isotonic recalibration.
• Conformal prediction — a distribution-free way (formalized by Vovk and popularized by Angelopoulos and Bates) to wrap any model's predictions in an interval with a finite-sample coverage guarantee, by reading a quantile of the model's past errors on a calibration set; the guarantee holds only under exchangeability, which distribution shift and OOD inputs break.
• Quantile regression / prediction interval — fitting the edges of the distribution (e.g. the 5th and 95th percentiles via the pinball loss) so a prediction arrives as a band rather than a point; the suite's viral_lrv.py brackets each log-reduction value with a 90% band and checks its empirical coverage.
• Applicability domain (AD) — a per-prediction gate that flags when an input lies outside the region the model was trained on, so the prediction is withheld or hedged rather than trusted blindly; in soft_sensor_pls.py it is a Hotelling T² and SPE check on each incoming spectrum.
• Autoencoder / VAE — unsupervised compress-and-reconstruct networks used for anomaly detection (high reconstruction error flags an out-of-family batch) and, for the VAE's probabilistic latent space, synthetic-sample generation.
• Bias-variance trade-off — the decomposition of expected error into being-too-simple (bias²), fitting-the-noise (variance), and irreducible noise; on small data, variance dominates, so the lower-variance model wins.
• Leave-one-batch-out CV / lineage grouping — a batch-grouped split in which every row sharing a lineage back to the same fed-batch run is held out together; the grouping key is Book 4's transitive bp:derivedFrom edge rooting each lot in the cell bank, so honest validation reads straight off the knowledge graph rather than off a remembered convention.
• SHACL input contract — validating a training set against the same closed-world bp:ReleaseShape (completeness, single-valuedness, declared range) that gates a lot at release, so the data a model fits on is proven complete and admissible before the fit, not just pinned by hash afterward.
• Locked model — a model frozen in production (weights, preprocessing, scaler, operating range, and the pinned software environment and seed), version-pinned and never edited in place; the only pattern draft Annex 22 permits for critical applications.
• PCCP (Predetermined Change Control Plan) — the pre-approved written specification of how a model may be retrained (data, fixed algorithm, acceptance criteria, rollback), so a retrain inside the envelope is a planned event rather than a new regulatory negotiation.
• GAMP 5 / CSA — the risk-based framework for validating computerized systems (GAMP 5) and the FDA's critical-thinking, risk-proportionate reframing of validation effort (Computer Software Assurance), both extended to ML by the ISPE GAMP AI guide.
• FDA 7-step model-credibility framework — state the question and context of use, assess model risk (influence × consequence), and gather credibility evidence proportional to that risk; "credible because evidence cleared a pre-stated gate," not "credible because it ran."
• Evidence tier / maturity — this book's two-axis grading of every claim: tier (peer-reviewed-independent, peer-reviewed-self-authored, vendor-self-reported, press-release-only) and maturity (production, pilot, research), kept separate so a vendor slide is never laundered into a fact.

Where this leads

We have an engine and a paradigm for trusting it: climb no higher up the model ladder than the data forces, and validate, lock, and document whatever you choose. But "which model" assumes we already know what we are predicting and why it counts as the right target. The next chapter, Target and Concept, steps back to that prior question — how a vague business goal ("make a better batch") becomes a precise, learnable target with a defined context of use, how the target product profile and CQAs pin down what "good" means, and how the choice of target silently determines everything the model can and cannot be trusted to do. The honest validation we framed here only means something once the target itself is the right one.