Models and Validation: From PLS to Transformers, Under GxP

📍 Where we are: Part I · Foundations of Learning in Bioprocess — Chapter 3. The last chapter built the fuel: data that is ready, contextualized, turned into leak-free features, and honestly split by batch. This chapter chooses an engine. We walk the ladder of model families from partial least squares to transformers, asking at each rung how much data it needs and how it must be validated to be trusted with a decision about a medicine — and we argue, against the prevailing hype, that the simplest engine usually wins.

A model is the function that turns the contextualized fuel of the previous chapter into a number a person can act on: a titer from a spectrum, a clone rank from a sequence, a pass/fail from a release panel. Newcomers to bioprocess machine learning arrive expecting the model choice to be the hard, exciting part — which architecture, how many layers, which optimizer. It is not. By the time you have read this chapter you should hold two convictions that run against the grain of the broader field. First, the model family that earns its keep in production bioprocess is overwhelmingly the simplest one that fits — a 40-year-old linear latent-variable method, not a transformer. Second, choosing the model is only half the job: under GxP a model is not "good" because its R² is high, it is trustworthy because it was validated, locked, and documented to a standard a regulator will accept. This chapter covers both halves, because in bioprocess they are inseparable.

We keep the same running example the whole series uses. The golden run BATCH-2026-001 carries an SEC monomer purity of 98.611%; its sibling BATCH-2026-004 goes OOS with a host-cell-protein result of 128 ng/mg against a 100 ng/mg spec maximum. The same in-line Raman spectra and offline assays from the simulator that fed Chapter 2 feed the models here, and the example suite in examples/platform/ml/ provides a runnable PLS-versus-deep head-to-head we will read together.

The simple version

Suppose you must learn to guess a cake's sweetness from its smell, and you have only six cakes to learn from. You could hire a world-class chef who has tasted a million cakes and can describe a thousand subtle aromas — but with six examples, that vast knowledge has nothing to grip; the chef will "learn" coincidences in your six cakes and be confidently wrong on the seventh. Or you could use a simple rule of thumb — "more vanilla smell, more sweet" — fit to your six cakes. The rule of thumb is less impressive but far more reliable, because it has less room to fool itself. That is the entire argument of this chapter's first half: a model with fewer ways to go wrong needs less data to go right, and bioprocess never has enough data to feed the fancy model. The second half adds the part no cake has: before anyone is allowed to act on the model's guess about a medicine, the guess — and the model that made it — must be proven, frozen, and written down, so that the guess you trust tomorrow is the one you proved today.

What this chapter covers

• The model families, and when each fits: linear regression, PLS, and PCA (the small-data workhorse); tree ensembles and gradient boosting; Gaussian processes with Bayesian optimization; and neural networks (MLP, 1D-CNN for spectra, autoencoders/VAE, transformers).
• Why deep learning rarely beats PLS on bioprocess data: the bias-variance trade-off, sample efficiency, extrapolation, and interpretability for review — with a runnable PLS-versus-CNN head-to-head that shows it.
• The GMP validation paradigm: locked versus continuously-learning models, the Predetermined Change Control Plan (PCCP), GAMP 5 and Computer Software Assurance (CSA) applied to ML, the ISPE GAMP AI guide, the FDA 7-step model-credibility framework, and EU draft Annex 22 — at the altitude this chapter needs, with the regulation chapter and MLOps chapter going deeper.
• The evidence-tier and maturity callout this book uses to grade every claim, formally introduced here.

How this book grades its evidence

Before we compare models, we fix the lens this book uses to judge every claim about them — because the literature on bioprocess ML is full of impressive numbers whose credibility varies wildly. From here on, every external claim carries two labels.

The first is an evidence tier, ranked from strongest to weakest:

• peer-reviewed-independent — published, peer-reviewed work whose authors are not the vendor or operator selling or running the thing. The gold standard.
• peer-reviewed-self-authored — peer-reviewed, but co-authored by the company that built or deployed the method. Credible on the science, but the framing favors the author.
• vendor-self-reported — a vendor's own claim (a white paper, a slide, a product page). Useful as a signal of direction; not evidence of a result.
• press-release-only — a number with no method behind it you can inspect. Treat as marketing.

The second is a maturity label for how far a method has actually traveled: (production) means deployed in a routine GMP plant; (pilot) means demonstrated at scale but not in routine production; (research) means a paper or a lab result. A claim can be high on one axis and low on the other — a peer-reviewed-independent result that is only (research), or a (production) deployment whose only public evidence is vendor-self-reported. Keeping the two apart is the whole point, and it is why this book refuses to launder a vendor slide into an established fact.

Evidence

This is the callout you will see throughout the book. Whenever a claim's credibility or maturity is contestable — a headline accuracy number, a "fewer experiments" figure, a "first-ever" deployment — it gets graded here, in its own block, so the tier travels with the claim and is never silently upgraded. The strongest single anchor for this chapter's central argument — that classical chemometrics beats deep learning on real bioprocess spectra — is peer-reviewed-self-authored and (pilot): the Boehringer Ingelheim study that predicted 16 quality attributes in-line during Protein A capture, whose best overall model was k-nearest-neighbors, not a deep network [1]. We return to it below.

The model families, and the question each one answers

There is no single best model. There is a ladder, ordered roughly by how much data and how much trust each rung demands, and the engineering skill is to climb no higher than your problem forces you to. Here is the ladder, rung by rung, with the bioprocess question each rung was built to answer.

Linear regression, PLS, and PCA — the workhorse

At the bottom sits the family that does the overwhelming majority of real production work in bioprocess: linear latent-variable methods. Principal Component Analysis (PCA) compresses many correlated variables into a few uncorrelated directions of greatest variance; it is the engine of the multivariate monitoring the release chapter builds. Partial Least Squares (PLS) is its supervised cousin: where PCA finds the directions of greatest variance in X, PLS finds the directions in X that best predict y. That distinction is exactly what a spectroscopic soft sensor needs. A Raman spectrum has 701 wavenumber channels that are massively collinear — neighboring channels move together — and only a handful of underlying chemical factors actually drive titer. PLS projects those 701 correlated channels onto a few latent components and regresses titer on them, sidestepping the catastrophe an ordinary least-squares fit would suffer on 701 collinear predictors with a few hundred rows.

PLS is roughly forty years old, it is the documented incumbent in commercial spectroscopic PAT, and it is genuinely hard to beat in the small-data regime [1][2]. The whole industry of in-line Raman and NIR soft sensors — glucose, lactate, titer, up to closed-loop glucose control in CHO culture — runs on PLS chemometrics, and the dominant commercial monitoring suites (Sartorius SIMCA, AspenTech ProMV) are productized PCA/PLS with the Hotelling's T² and SPE charts the release chapter dissects [2] (production). When in doubt in bioprocess, you start here — and you very often stay here.

Plain linear regression sits one step below PLS and is worth keeping in view, because the regularized linear models are themselves underrated bioprocess workhorses. Ridge regression (an L2 penalty) tames collinearity by shrinking coefficients toward zero; LASSO (an L1 penalty) does feature selection by driving some coefficients exactly to zero, which is precisely what you want when a few of many metabolite features actually matter. The release predictor uses an L2-regularized logistic regression for exactly the reasons this family is favored: it is calibrated, its standardized coefficients read directly as log-odds an investigator can interpret, and it has almost no room to overfit a few hundred rows. The common thread across linear regression, ridge/LASSO, PLS, and PCA is low variance and high transparency — the two properties small-data GMP work prizes above raw flexibility.

Tree ensembles and gradient boosting — tabular, nonlinear, interpretable enough

When the data is tabular (process parameters, metabolite summaries, categorical media or clone identifiers) rather than spectral, and the relationship is nonlinear or full of interactions, the next rung is tree ensembles. A single decision tree splits the feature space into boxes; a random forest averages many trees grown on bootstrapped samples to cut variance; gradient boosting (XGBoost, LightGBM) grows trees sequentially, each correcting the last one's residual, which is usually the strongest tabular learner available. Their bioprocess home is the release predictor and the manufacturing-operations models — predicting an outcome from a few dozen engineered features. They handle nonlinearity and mixed feature types out of the box, they are robust to feature scaling, and — crucially for review — they emit feature importances and partial-dependence plots, so an investigator can see which feature drove a prediction. They are not magic on small data: a boosted ensemble can overfit a few hundred rows as eagerly as a neural net, and they extrapolate poorly (a tree cannot predict outside the range of values it saw). But on tabular, moderate-sized bioprocess data they are a sensible, defensible default.

Gaussian processes and Bayesian optimization — when each experiment is precious

The third rung is the one built for bioprocess's defining scarcity: experiments cost weeks and a fortune, so you want a model that tells you where to run the next one. A Gaussian Process (GP) is a model that returns, at every point in the input space, not just a predicted mean but a calibrated uncertainty — small where you have data, large where you do not — governed by a kernel (commonly Matérn or squared-exponential) that encodes how similar two settings' outcomes should be. That uncertainty field is what powers Bayesian Optimization (BO): fit a GP to the experiments run so far, use an acquisition function (Expected Improvement, or its multi-objective cousin Expected Hypervolume Improvement when you must trade titer against quality) to pick the next experiment that best balances exploiting the current best region against exploring uncertain ones, run it, refit, repeat. The result is a feedback loop that reaches a competitive optimum in materially fewer runs than a fixed factorial design-of-experiments grid — the engine behind process development and media optimization, supported across recent peer-reviewed bioprocess work (research, peer-reviewed-independent) [3]. GP-BO is the rare case where a more sophisticated model is exactly what small data calls for, because its sophistication is spent on quantifying ignorance rather than on raw capacity, and that honest error bar is itself a governance asset — a model that says "I don't know here" is far easier to trust than one that guesses confidently. Its cost is poor scaling — classical GPs are cubic in the number of training points — which is precisely no obstacle when you have thirty experiments.

Neural networks — MLP, 1D-CNN, autoencoders/VAE, transformers

At the top of the ladder sit the neural networks, in roughly increasing order of appetite for data:

• A multilayer perceptron (MLP) is a stack of fully-connected layers — a flexible nonlinear regressor. It can fit anything given enough data; on a few hundred bioprocess rows it mostly fits noise.
• A 1D-CNN treats a spectrum as a one-dimensional signal and learns local spectral features with shared convolutional filters. This is the architecturally correct deep model for Raman or NIR — it respects the structure of a spectrum the way a 2D-CNN respects an image — and it is the deep model we benchmark against PLS below.
• Autoencoders and variational autoencoders (VAE) are unsupervised: they compress data to a low-dimensional latent code and reconstruct it, learning "normal" structure. Their bioprocess use is anomaly detection (a high reconstruction error flags an out-of-family batch, a neural cousin of MSPC's SPE statistic) and, for the VAE, generating plausible synthetic samples.
• Transformers are the attention-based architecture behind large language models. On bioprocess time series they remain largely (research); where transformers genuinely earn their place in this domain is on sequence and text data — protein-language models for molecule and clone work, and LLMs over documents in the generative-AI chapter — not on a six-batch fed-batch dataset.

That ordering matters because it is also, almost exactly, the order in which a model's data appetite grows and its defensibility shrinks. Which brings us to the central argument of the chapter.

Why deep learning rarely beats PLS on bioprocess data

The single most common mistake a newcomer makes in bioprocess ML is to reach for deep learning because it is powerful, and to be surprised when it loses to a linear model from the 1980s. This is not bad luck or bad tuning; it is structural, and it has four interlocking reasons.

The bias-variance trade-off, in plain terms. Every model's error decomposes into bias (error from being too simple to capture the truth) and variance (error from being so flexible that it fits the noise in this particular training sample). A high-capacity model — a deep net with hundreds of thousands of parameters — has low bias but enormous variance: with few training examples, it has the freedom to fit the idiosyncrasies of those examples and generalizes poorly. PLS, with a handful of latent components, has higher bias but far lower variance. On small data, variance dominates the error, so the lower-variance model wins. This is not a knock on deep learning; it is the bias-variance trade-off doing exactly what the textbook says it does, in the regime where bioprocess lives.

Sample efficiency, and how little data there really is. Recall the cold-start reality: the binding constraint is not the number of rows but the number of independent batches, which grows by ones, slowly, at the cost of weeks each. A deep network's parameter count routinely exceeds its training-example count by orders of magnitude. PLS's effective complexity is a few latent components. Deep learning's spectacular wins came from datasets of millions of examples (ImageNet, web-scale text); bioprocess offers six batches. There is no architecture trick that manufactures information the data never contained — a lesson the hybrid-modeling chapter turns into a positive strategy by injecting mechanistic knowledge the data never had to provide.

Extrapolation, where manufacturing actually operates. A validated process runs in a tight, characterized window; the moments you most need a model are excursions and edges — exactly where there is no training data. A linear model extrapolates predictably (you can see where it is heading and bound the error), and a GP tells you honestly that its uncertainty has exploded. A deep net extrapolates unpredictably and confidently — it returns a crisp number with no signal that it has wandered off the map. In a domain where a confidently-wrong prediction can mis-feed a culture, predictable extrapolation is worth more than raw in-distribution accuracy.

Interpretability for review. This reason has no analogue in consumer ML and dominates in pharma. A PLS model exposes its regression coefficients and variable-importance (VIP) scores; an investigator can point at the wavenumbers driving a prediction and check them against known chemistry — a glucose band, a protein amide signature — and confirm the model is right for the right reason rather than exploiting a spurious correlation. A deep net is, to a reviewer, a black box — and a black box is a hard thing to defend in front of a regulator who must understand why a model decided what it did about a medicine. Post-hoc explainability tools (SHAP values, saliency maps) help, but they are approximations of an opaque model's behavior, not the model's actual reasoning, and a reviewer can reasonably distrust an explanation that the model itself did not produce. The hybrid models chapter makes the same point from the other side: a model whose prediction can be decomposed into "what the physics said" and "what the data added" slots into validation far more comfortably than an opaque one. Interpretability is not a nicety here; it is a gating requirement for GMP use, and it is the reason the choice of model and its validatability cannot be separated.

The honest corrective the field repeatedly needs: deep learning is not generally superior on small bioprocess data. The most-cited apparent counterexample — the Boehringer Ingelheim work predicting 16 quality attributes in-line during Protein A capture in roughly 38 seconds — is genuinely excellent and genuinely (pilot), but its best overall model was k-nearest-neighbors, a classical distance-based method with no neural network in it [1]. It is frequently miscited as proof of a "deep-learning Raman wave"; it is, if anything, the strongest single piece of evidence for classical chemometrics on real spectra. Where head-to-head deep-versus-PLS Raman benchmarks exist, deep learning typically matches rather than decisively beats PLS, at a large cost in data, compute, and reviewability [1][4] (pilot/research). The practitioner's rule that falls out of all four reasons is blunt: start at the bottom of the ladder and climb only when the data forces you to — which, in bioprocess, is rarely.

The model ladder mapped onto data appetite: the methods that do real production bioprocess work — PLS/PCA, tree ensembles, and GP-BO — sit inside the small-data band where bioprocess actually lives, while MLPs, CNNs, and transformers sit to its right, needing far more data than six batches can supply; climb the ladder only when the data forces you to. Original diagram by the authors, created with AI assistance.

The head-to-head, in code

The example suite makes this argument runnable rather than rhetorical. Two modules fit the same titer-from-Raman problem on the same golden-batch spectra: soft_sensor_pls.py (a six-component PLS) and soft_sensor_deep.py (a compact 1D-CNN). The CNN treats each 701-channel spectrum as a one-dimensional signal, learns local spectral features with shared filters, and regresses titer through a small dense head — the architecturally correct deep model for a spectrum. The point of running them side by side is not to crown a winner but to show, on real numbers, that orders of magnitude more parameters buy no better held-out result on small, clean spectra.

# soft_sensor_deep.py — a compact 1D-CNN over the 701-wavenumber Raman signal,
# benchmarked head-to-head against the PLS baseline in soft_sensor_pls.py.
class SpectraCNN(nn.Module):
    """Two conv blocks over wavenumber, then a small dense head."""
    def __init__(self, n_wavenumbers: int):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv1d(1, 8, kernel_size=15, padding=7), nn.ReLU(),
            nn.MaxPool1d(4),
            nn.Conv1d(8, 16, kernel_size=11, padding=5), nn.ReLU(),
            nn.AdaptiveAvgPool1d(8),
        )
        self.head = nn.Sequential(
            nn.Flatten(), nn.Linear(16 * 8, 32), nn.ReLU(),
            nn.Dropout(0.2), nn.Linear(32, 1),
        )

    def forward(self, x):              # x: (batch, 1, n_wavenumbers)
        return self.head(self.features(x))


if __name__ == "__main__":
    from soft_sensor_pls import train_pls
    pls = train_pls()                  # the 40-year-old chemometric baseline
    cnn = train_cnn()                  # ~thousands of parameters of deep model
    print(f"  PLS    : R2={pls['r2']}  ({pls['n_params']} coefficients)")
    print(f"  1D-CNN : R2={cnn['r2']}  ({cnn['n_params']} parameters)")
    print(f"  PLS uses {cnn['n_params'] / pls['n_params']:.0f}x fewer params "
          f"and is not beaten on R2.")

Running the head-to-head prints the lesson directly (numbers illustrative, from the simulated golden-batch spectra):

Head-to-head: titer from 701-wavenumber Raman, golden batch BATCH-2026-001
  PLS    : R2=0.9847  RMSE=0.21 g/L  (702 coefficients)
  1D-CNN : R2=0.9731  RMSE=0.28 g/L  (5489 parameters)
  PLS uses 8x fewer params and is not beaten on R2.   # illustrative

The CNN is not broken — it is a perfectly reasonable model, and on a clean simulated signal it lands close. That is exactly the point. With several times the parameters, an extra dependency on PyTorch, far more compute, and a model no reviewer can read, the deep net fails to beat a linear method from the 1980s on the metric that matters. On real Raman, with its scatter, fouling, and run-to-run drift, the deep model's higher variance usually makes the gap worse, not better, while PLS's reviewable coefficients keep it defensible. Note one honest caveat the data chapter insists on: this demonstration trains within a single batch to compare architectures cleanly; a deployable soft sensor must be validated under the batch-grouped split, which is the harness that makes any of these numbers admissible.

Anatomy of a validated PLS soft-sensor package

A model that earns a place in a GMP plant is not a .pkl file with a good R² — it is a validated package, and what travels alongside the fitted coefficients is what makes it usable for a decision about a medicine. Dissect the package the way a quality reviewer would, and the whole second half of this chapter is laid out as fields.

One validated soft-sensor package, fully unpacked: the build provenance that pins it to an exact dataset hash, scaler, and preprocessing; the green validation core with acceptance criteria, operating range, and advisory scope; the interpretability artifacts (PLS coefficients and VIP scores) that a deep net cannot supply; the locked-model and PCCP lifecycle; and the GAMP 5 / CSA / FDA-credibility governance that makes it a validated object rather than a model file. Original diagram by the authors, created with AI assistance.

Read the card top to bottom and the validation paradigm is concrete. The build block is provenance: the feature contract, the SNV preprocessing pinned as part of the model (preprocessing is the model — fit it on training data only, or every prediction leaks), the fitted scaler that must travel with the coefficients, the six latent components, and the training dataset pinned by its sha256 so "which data trained this?" is never a guess. The green core is what validation produced: held-out R² and RMSE against written acceptance criteria, the qualified operating range, and the intended-use scope — advisory, human decides. The amber block is the advantage that justified choosing PLS in the first place: regression coefficients and VIP scores a reviewer can read against known chemistry. The rose lifecycle block holds the locked status, the PCCP that governs future change, and the next revalidation date. The governance panel carries the GAMP 5 category, the CSA risk assessment, the FDA credibility tier, and the four-eyes signatures. A model file has weights; a validated package has all of this — which is the only reason it is allowed near a batch.

The GMP validation paradigm

The second half of "which model" is "how do you prove you can trust it." In consumer ML the answer is a held-out test score. Under GxP it is a far larger apparatus, and understanding its shape is as important as understanding the models.

Locked versus continuously-learning models. A GMP process must be validated — proven, frozen, and held under change control so that what you make tomorrow is what you proved yesterday. A model that keeps learning is, by definition, a system that keeps changing, which is the thing validation forbids without re-qualification. The resolution the industry and regulators have converged on is the locked model: weights, preprocessing, scaler, and operating range all version-pinned and unchangeable in place. It does not learn on the fly. The MLOps chapter builds the full lifecycle around this; here it is enough to state the rule that governs every model in this book — locked-then-relearn, never continuously-learning, for anything touching a critical quality attribute.

The Predetermined Change Control Plan (PCCP). A locked model that can never change would be a dead end, because the world moves and the model drifts. The PCCP is the mechanism that lets a model change on purpose without a fresh regulatory negotiation each time: a pre-approved, written specification of how the model may be retrained — which data, which fixed algorithm and hyperparameters, what acceptance criteria the new version must clear, and the rollback plan. A retrain that stays inside the PCCP's envelope is a planned, documented event rather than an unforeseen change. The PCCP is the bridge across the validation-versus-learning gap, and the MLOps chapter shows it driving a real retraining loop.

GAMP 5 and Computer Software Assurance (CSA). Pharma already had a discipline for trusting software before ML arrived: GAMP 5 (Good Automated Manufacturing Practice), the risk-based framework for validating computerized systems, which categorizes software by how custom and how critical it is and scales the validation effort to match. Computer Software Assurance (CSA) is the FDA's 2022-era reframing of that effort — a deliberate shift away from exhaustive, document-everything testing toward critical thinking and risk-based assurance, spending validation effort where the patient risk is, not uniformly. Applied to ML, GAMP 5 and CSA say: a soft sensor that only advises a human carries less risk — and a lighter assurance burden — than one wired to act on a CQA, and the evidence you gather should be proportional to that risk. The ISPE GAMP AI guide extends this established computerized-system-validation thinking specifically toward AI/ML — risk-based, lifecycle-oriented, demanding ongoing evidence rather than a one-time test — translating "validate your software" into "validate your model and monitor it forever" [5].

The FDA 7-step model-credibility framework. The FDA's guidance on the credibility of computational models gives the risk-proportionate spine. In outline, the seven steps are: state the question of interest; define the model's context of use (exactly what role it plays in the decision); assess the model risk — the product of how much the decision relies on the model (model influence) and how severe being wrong would be (decision consequence); plan the credibility activities proportional to that risk; execute them; document the results; and decide on adequacy for the stated use, re-assessing whenever the use changes. The deep idea is simple and powerful: a model is not trustworthy because it ran; it is trustworthy because evidence was produced and checked against a pre-stated acceptance criterion, at a rigor matched to the consequence of being wrong [6]. A soft sensor that merely advises sits low on the risk axis and needs lighter evidence; a model wired to release a lot sits at the top and needs the heaviest. The example suite's run_all.py harness is a small software analogue of exactly this — it runs every model in the suite, captures whether each cleared its own pre-stated acceptance gate (the module's assert), and records the SHA-256 of the dataset it was fitted on, emitting a machine-checkable credibility ledger rather than a benchmark. Passing the gate is necessary, not sufficient; credibility also needs documented intended use, change control, and human oversight.

# run_all.py — a model is credible because EVIDENCE cleared a pre-stated gate
# on a PINNED dataset, not because it ran. The script analogue of the FDA framework.
@dataclass(frozen=True)
class ModelEvidence:
    module: str            # the model that produces the evidence
    gate: str              # the pre-stated acceptance criterion (its own assert)
    datasets: tuple        # the data it was fitted on, pinned by sha256
    passed: bool | None    # did the evidence clear the gate?

LEDGER = [
    ModelEvidence("soft_sensor_pls.py",  "Raman->titer R2 > 0.85 on held-out hours",
                  ("raman_spectra.parquet",)),
    ModelEvidence("mspc.py",             "MSPC flags ONLY the OOS batch; SPE points at HCP",
                  ("hplc_results.csv",)),
    # ... every module ends in an assert == an acceptance criterion in a protocol.
]

EU draft Annex 22, at altitude. The EU's draft GMP Annex 22 draws the hardest line of all: it requires AI models used in critical GMP applications to be static (locked), and explicitly excludes self-learning, generative, and adaptive AI from those critical uses, demanding a predetermined change-control approach for any update [7]. In other words, the draft codifies locked model plus PCCP as the only acceptable pattern where a model touches quality — the continuously-learning model is, for now, regulatorily off the table for critical decisions. (The regulation-focused chapters go deeper on Annex 22 and the FDA framework; this chapter establishes only the shape every model in the book must fit.)

Model monitoring, explainability, and documentation. Three expectations run through all of the above and recur in every later chapter. Monitoring: a validated model is watched for drift after deployment, because its performance can decay silently as the living process and its hardware move (the MLOps chapter builds the detectors). Explainability: a model used for a GMP decision must be interpretable enough that a human can understand and defend its reasoning — the single largest practical reason PLS's reviewable coefficients beat a deep net's opacity in this domain. Documentation: intended use, training data provenance, validation evidence, acceptance criteria, operating range, and change-control history are not paperwork about the model; under GxP they are the model, the difference between a .pkl file and a validated object.

The unsolved part: a static model watching a moving process

The tension this chapter sets up but does not resolve is the one the whole back half of the book wrestles with: we have just argued that a credible GMP model must be locked, and the data chapter argued that the process it watches is alive and never identical twice. A frozen model aimed at a moving target is, by construction, slowly going wrong — its accuracy decays the moment the cell line adapts, a probe fouls, or a new raw-material lot arrives. Validation freezes the model precisely when the process refuses to hold still. There is no clean dissolution of that paradox; there is only management of it — lock the model, monitor it for drift, retrain off-line into a new validated version under a PCCP, and promote through a human gate. That management loop, the detectors that make it run, and the honest limits of detecting drift before the slow reference data lands are the subject of the MLOps and lifecycle chapter. For now, the lesson is that choosing the model and keeping it true are two halves of one problem, and the validation paradigm exists to hold them together.

What this chapter adds to the model suite

This chapter does not contribute a new module so much as frame the suite the data chapter bootstrapped and the later chapters fill in. Two existing modules anchor its argument:

• soft_sensor_pls.py and soft_sensor_deep.py — the PLS-versus-1D-CNN head-to-head on the golden-batch Raman spectra. Same data, same target, two rungs of the ladder; the deep model, with several times the parameters, does not beat the 40-year-old linear baseline on the held-out number, making the chapter's central claim runnable rather than rhetorical.
• run_all.py — the model-credibility evidence harness. It is deliberately not another model; it runs every module in the suite, checks each one's evidence against its own pre-stated acceptance gate, and pins the dataset by SHA-256 — a software-assurance analogue of the FDA 7-step framework, demonstrating in code that credibility is evidence against a fixed criterion on pinned data, not a good run. Every later chapter's module ends in an assert precisely so this harness can read it as an acceptance criterion.

Together they make the chapter's two halves concrete: which engine fits (PLS, on the evidence), and what it takes for that engine to be trusted (a fixed gate cleared on pinned data).

Why it matters

The fastest way to waste a bioprocess ML project is to spend it on the wrong half of the problem. Newcomers pour effort into model architecture — deeper nets, fancier optimizers, the latest transformer — and discover that on six batches the elaborate model loses to PLS, costs more, needs a GPU, and cannot be explained to a reviewer. Meanwhile the half that actually decides whether the model ever reaches a plant — validation, locking, documentation, the credibility evidence a regulator demands — gets treated as an afterthought, and the model dies in review. This chapter inverts both instincts. Climb no higher up the model ladder than the data forces you to, because in bioprocess the simplest engine that fits is almost always the one that ships; and treat validation not as paperwork bolted on at the end but as a first-class design constraint that shapes the model choice from the start — which is exactly why interpretability pushes you toward PLS before you have run a single fit. Get those two right and the model is both good and trustworthy; get either wrong and a high R² is worth nothing.

In the real world

The production reality matches the argument with unusual cleanliness. The strongest deployed bioprocess ML is overwhelmingly classical: in-line Raman/NIR soft sensing on PLS chemometrics for glucose, lactate, and titer, up to closed-loop glucose control in CHO culture (production); and PCA/PLS multivariate monitoring — Sartorius SIMCA, AspenTech ProMV — for continued process verification and golden-batch monitoring (production) [2]. GP-BO is real and growing in process development (research/pilot, peer-reviewed-independent) [3]. Deep learning's genuine production foothold is narrow and specific — automated visual inspection of vials, a computer-vision problem with abundant images, which the fill-finish chapter covers — not soft sensing on small spectral or kinetic datasets, where it repeatedly fails to beat the chemometric baseline. The most-cited apparent deep-Raman success was, on inspection, KNN [1]. And the validation paradigm is not aspirational: the ISPE Pharma 4.0 surveys consistently find ML clustered in monitoring and advisory roles, almost never in autonomous control, precisely because the locked-model-plus-PCCP lifecycle and the credibility evidence are hard, expensive, and not something you can pip install. The honest verdict the rest of the book elaborates: the engine is usually simple, and the trust is always expensive.

Key terms

• PLS (Partial Least Squares) — the supervised latent-variable method that projects many collinear predictors (e.g. 701 Raman channels) onto a few components that best predict the target; the small-data workhorse of bioprocess soft sensing.
• PCA (Principal Component Analysis) — the unsupervised cousin that finds directions of greatest variance; the engine of multivariate monitoring (MSPC).
• Tree ensembles / gradient boosting — random forests and boosted trees; the strong, reasonably interpretable default for tabular, nonlinear, moderate-sized bioprocess data.
• Gaussian Process (GP) — a model that returns a calibrated uncertainty at every input; the surrogate that powers Bayesian optimization.
• Bayesian Optimization (BO) — the feedback loop that uses a GP's uncertainty to choose the next experiment, reaching an optimum in far fewer runs than a fixed DoE grid; built for bioprocess's experiment scarcity.
• 1D-CNN — a convolutional network that treats a spectrum as a one-dimensional signal; the architecturally correct deep model for Raman/NIR, and the one that still fails to beat PLS on small data.
• Autoencoder / VAE — unsupervised compress-and-reconstruct networks used for anomaly detection (high reconstruction error flags an out-of-family batch) and synthetic-sample generation.
• Bias-variance trade-off — the decomposition of model error into being-too-simple (bias) and fitting-the-noise (variance); on small data, variance dominates, so the lower-variance model wins.
• Locked model — a model frozen in production (weights, preprocessing, scaler, operating range), version-pinned and never edited in place; the only pattern EU draft Annex 22 permits for critical applications.
• PCCP (Predetermined Change Control Plan) — the pre-approved written specification of how a model may be retrained, so a retrain inside the envelope is a planned event rather than a new regulatory negotiation.
• GAMP 5 / CSA — the risk-based framework for validating computerized systems (GAMP 5) and the FDA's critical-thinking, risk-proportionate reframing of validation effort (Computer Software Assurance), both applied to ML.
• FDA 7-step model-credibility framework — state the question and context of use, assess model risk, and gather credibility evidence proportional to that risk; "credible because evidence cleared a pre-stated gate," not "credible because it ran."
• Evidence tier / maturity — this book's two-axis grading of every claim: tier (peer-reviewed-independent, peer-reviewed-self-authored, vendor-self-reported, press-release-only) and maturity (production, pilot, research), kept separate so a vendor slide is never laundered into a fact.

Where this leads

We have an engine and a paradigm for trusting it: climb no higher up the model ladder than the data forces, and validate, lock, and document whatever you choose. But "which model" assumes we already know what we are predicting and why it counts as the right target. The next chapter, Target and Concept, steps back to that prior question — how a vague business goal ("make a better batch") becomes a precise, learnable target with a defined context of use, how the target product profile and CQAs pin down what "good" means, and how the choice of target silently determines everything the model can and cannot be trusted to do. The honest validation we framed here only means something once the target itself is the right one.