Generative AI and LLMs: Copilots, CAPA, and the Limits of Agents

📍 Where we are: Part VI · The Whole System — Chapter 24. The previous chapter, Manufacturing Operations, put learning into the plant's nervous system — predictive maintenance, yield analytics, and scheduling. This one turns to the layer that learns language: the deviation reports, batch records, SOPs, and CMC documents that wrap every batch. It is the loudest part of the 2023-2026 AI wave, and the part where the gap between demo and deployment is widest.

Every chapter so far has learned a number — a titer, a log-reduction value, an anomaly score, a maintenance horizon. This chapter learns words. Biomanufacturing runs on an enormous, mostly unstructured paper trail: a deviation is a free-text narrative; a CAPA (corrective and preventive action) is an investigation written in prose; a batch record is hundreds of pages of steps, entries, and signatures; a CMC (chemistry, manufacturing, and controls) section is a regulatory document drafted by hand over weeks. Large language models (LLMs) are, for the first time, genuinely good at reading and drafting exactly this kind of text — and the result has been a flood of "copilots," "agents," and "assistants" pitched at quality and manufacturing since GPT-4 landed in 2023 [1][2].

The honest reading is the one this whole book keeps arriving at, sharpened here to its hardest edge. Generative AI in GMP (Good Manufacturing Practice — the binding, in-force rules for how a regulated drug must be made; GxP is the umbrella term for the whole family of "Good x Practice" regulations, of which GMP is the manufacturing member; cGMP is the FDA's "current" GMP, "current" meaning you must follow today's accepted practice, not a frozen old standard) is real, useful, and almost entirely advisory: it triages, retrieves, summarizes, and drafts, and a qualified human reviews and signs. Where a firm let it cross that line — generating specifications and master production records without quality-unit review — the FDA (the US Food and Drug Administration, the agency that regulates and inspects drug manufacturers) issued its first AI-citing cGMP warning letter (an official, public enforcement notice that a manufacturer is violating the regulations and must correct them — a serious sanction, not a routine memo) (Purolea, 2 April 2026) [3][4]. And the draft EU/PIC/S GMP Annex 22 — an annex (a topic-specific appendix) to the EU's Good Manufacturing Practice rules, co-issued with PIC/S, the international Pharmaceutical Inspection Co-operation Scheme that harmonizes GMP inspection across member regulators — draws the boundary in regulation: it excludes generative and continuously-learning AI from critical GMP decisions outright [5][6]. This chapter is the map of what works, what does not, and exactly where the line falls.

The simple version

Think of a brilliant, fast new analyst who has read every deviation report your site ever filed but has no signing authority. You hand them a new problem; in seconds they say "this looks like the three temperature excursions we had last spring — here are the investigations and the CAPAs that closed them, and here is a first draft of your write-up." That is genuinely valuable: it saves hours and stops you reinventing the wheel. But the analyst sometimes states things with total confidence that are simply wrong (a "hallucination"), and they are not allowed to decide anything — to release a batch, close a CAPA, or approve a record. A qualified person reads what they produced, checks it against the real evidence, and signs. Generative AI in a GMP plant is exactly that analyst: it accelerates the human, it does not replace the signature.

What this chapter covers

• Where the GenAI wave is actually real in manufacturing and quality: deviation/CAPA triage and drafting, NLP investigation assistants, MES/SOP/batch-record copilots, CMC and regulatory drafting, and knowledge management.
• Retrieval-Augmented Generation (RAG): why grounding an LLM in your own validated documents is the architecture that makes it usable in GMP, how the retriever and generator are actually built, and how you evaluate both.
• The two NLP tasks underneath every copilot: classification/triage and similarity retrieval — shown as runnable, transparent code in examples/platform/ml/deviation_triage.py, not a black box.
• The named products, sorted by maturity and evidence tier: Aizon, ValGenesis, Veeva, Mareana, Microsoft/Accenture/Capgemini copilots — what they claim and how strong the evidence is.
• The hard limits: hallucination, GxP validation of a non-deterministic model, and data leakage/confidentiality.
• The regulatory boundary: the Purolea warning letter as enforcement, and draft Annex 22's exclusion of generative AI from critical decisions — and exactly where agentic AI is therefore confined.

The GenAI wave, sorted into what is real

The marketing makes everything sound equivalent. It is not. Sort the deployments by what they actually do with a regulated decision, and a clear hierarchy appears — from genuinely useful and low-risk to overhyped and forbidden. Read each tier with two labels attached: a maturity (research / pilot / production) and an evidence tier (peer-reviewed-independent at the top, then peer-reviewed-self-authored, then vendor/consultancy-self-reported, then press-release-only at the bottom — "self-authored" ranks below "independent" because a study run by the company being studied has an interest in a flattering result, so an outside check carries more weight) — the same grading the book's case-study ledger applies, and the discipline that separates a fact from a slide.

Tier 1 — retrieve and summarize (real, widely piloted). The safest and most valuable use is turning a mountain of existing text into a fast answer. A deviation investigator asks "have we seen this failure before?" and the system retrieves the most similar historical deviations and their closed CAPAs. McKinsey reports a life-sciences manufacturer synthesizing roughly 70% of deviations and producing a first-draft CAPA for over 80% of cases with generative AI — a vendor/consultancy-reported figure, not peer-reviewed, and explicitly a drafting aid with human review [7]. MSD (Merck & Co.) has publicly described a RAG-based deviation assistant built on AWS Bedrock (Amazon's cloud service for running LLMs behind your own access controls) — with Amazon OpenSearch as the vector store (the searchable database of document vectors) over historical deviations — as exploratory, retrieval-not-predictive work [8]. This tier is mostly (pilot), occasionally edging into production for non-critical documentation.

Tier 2 — extract and classify (real, the NLP under the copilot). Beneath the chat interface sits old-fashioned, defensible NLP: classify a deviation into a category and route it; extract entities (equipment ID, batch, date, failure mode) from a narrative; flag a record as likely-recurring or likely-critical for prioritization. The peer-reviewed anchor here is a Merck & Co. study evaluating GPT-3.5, GPT-4, and Claude-2 (specific commercial LLMs — GPT from OpenAI, Claude from Anthropic) on real manufacturing deviation text: high-accuracy entity extraction (pulling structured facts — equipment ID, batch, failure mode — out of the prose), root-cause extraction (the underlying reason the failure occurred), and semantic retrieval (finding passages by meaning, not just shared keywords) of similar historical deviations, but with an explicit, named tension the authors call the "interplay between apparent reasoning and hallucination" — and a hedge that human review "might be necessary, especially in high-risk tasks" [1]. It is peer-reviewed but self-authored (Merck's own digital-services group), which is one tier below independent — still the most rigorously studied corner of the wave, and where this chapter's runnable example lives.

Tier 3 — draft regulated content (real but tightly governed). CMC drafting, batch-record review assistance, product-quality-review (PQR) generation, and SOP authoring. A PQR — the product quality review mandated annually for every product under 21 CFR 211.180(e) (Title 21 of the US Code of Federal Regulations — the legally binding US drug-manufacturing rulebook — section 211.180(e); later citations like 21 CFR Part 11 are sections of the same code) and EU GMP Chapter 1 — is a templated summary of a year of batch history (yields, deviations, OOS (out-of-specification) results, stability, complaints, changes), exactly the kind of high-volume, formulaic synthesis over existing records that GenAI drafts well and a human must still verify and sign. Sanofi has reported generating product quality reviews roughly eight times faster, targeting around 5,000 reports per year — a self-reported figure that must be labeled as such, not stated as established fact, and read as an aspiration rather than an audited result [9]. Vendors are racing here: Aizon (GxP manufacturing intelligence; the intelligent batch record with review-by-exception — a human reads only the records that flag a problem, not every entry), ValGenesis (validation lifecycle and tech-transfer drafting), and Veeva (Vault Quality AI agents, on a 2026 roadmap, running on Anthropic Claude and Amazon models via Bedrock) all sell into this tier, alongside horizontal copilots from Microsoft (Copilot over quality systems) and integrators like Accenture and Capgemini [10][11][12][13]. Every output here is a draft that a qualified person reviews, edits, and signs.

Tier 4 — agentic / autonomous (overhyped, and where the line is drawn). "Agentic AI" — systems that plan and take multi-step actions with minimal human input — is the loudest 2025-2026 pitch (Aizon pre-announced agentic capabilities including conversational generation of batch-release cockpits and PQR templates; Veeva's Quality and Safety agents are a roadmap item) [14][12]. In a GMP plant this tier is confined, by both enforcement and draft regulation, to non-critical, human-in-the-loop tasks. Cross the line into critical GMP — generating the records that govern how a batch is made or released, without quality-unit review — and you get the Purolea warning letter [3][15].

The pattern across all four tiers is the ISPE Pharma 4.0 reality stated throughout this book: AI/ML has the most pilots and the fewest scaled implementations, and production clusters in monitoring, vision, and human-in-the-loop documentation — not autonomous control of quality [2].

Retrieval-Augmented Generation: why grounding is the whole game

A raw LLM answers from its training data — a frozen, opaque snapshot that contains none of your SOPs, none of your deviation history, and no guarantee of currency. Ask it about your process and it will either decline or, worse, confidently invent a plausible answer. Retrieval-Augmented Generation (RAG) is the architecture that fixes this, and it is the single most important idea for making LLMs usable in a regulated plant.

RAG separates two jobs. A retriever searches your own validated corpus — SOPs, batch records, prior deviations, CMC sections — for the passages most relevant to the question, using vector similarity over text representations — each passage is turned into a vector (a list of numbers) so that similar text lands nearby and closeness can be measured, the mechanism the next section opens up in full. A generator (the LLM) is then prompted with the question plus those retrieved passages and instructed to answer only from them, citing which document each claim came from. The benefits are exactly the ones GMP demands:

• Grounding cuts hallucination. The model is told to answer from supplied, real documents rather than from memory, and to say "not found" when the passages do not support an answer.
• Traceability. Every statement can cite its source document and section — the audit trail a reviewer needs.
• Currency and access control. The corpus is your current, controlled documents; updating an SOP updates the system's knowledge without retraining, and document-level permissions can keep a model from surfacing text a user is not cleared to see.

RAG does not eliminate hallucination — a model can still misread or over-generalize a retrieved passage — but it converts an open-ended generation problem into a grounded, citable one, which is the difference between "interesting" and "auditable." This is why nearly every defensible GenAI deployment in the wave above is RAG-shaped: MSD's deviation assistant, Microsoft Copilot over quality systems, and the deviation/CAPA copilots all stand on a retriever over the firm's own documents [8][11][7].

The retriever is also the part you can build, test, and validate with classical, transparent tools — no giant model required. That is what the example below shows: the retriever and the triage classifier are ordinary, inspectable machine learning, and the generative LLM sits on top of them as the drafting layer that a human always reviews.

How a RAG retriever is built and evaluated

It pays to open the retriever's hood, because "vector similarity" hides a real engineering pipeline and a measurable contract — and because the retriever is the part of a GMP copilot you can actually validate. The full pipeline has five stages.

1. Chunking. A 300-page batch record is useless as a single blob; you split the corpus into passages (a deviation paragraph, an SOP step, a CMC subsection) so a retrieved hit is small enough to ground a specific claim and to cite precisely. Chunk too large and a citation points at a page; too small and the passage loses the context that made it relevant.
2. Embedding. Each chunk is turned into a vector that places semantically similar text near each other. Two honest choices sit on a spectrum. Sparse / lexical representations — TF-IDF, where each term is weighted by how often it appears in this document (term frequency) against how rare it is across the corpus (inverse document frequency) — are fully inspectable: you can read off which words drove a match. (The vector is just that list of per-word weights: one slot per distinct word in the corpus, mostly zeros, with a number in the slots for the words this chunk actually uses — "sparse" because so few slots are filled.) Dense / neural embeddings (a sentence-transformer, a neural model that turns a whole sentence into a vector) capture paraphrase and synonymy a plain word-count vector — a "bag of words," which ignores order and meaning — misses ("temperature excursion" near "thermal deviation") but are opaque. The shape of the pipeline is identical either way, which is why this chapter teaches the lexical version.
3. Indexing. The vectors go into a store (a vector store — a database built to find the vectors closest to a query vector, the role Amazon OpenSearch plays in MSD's assistant) that answers nearest-neighbor queries fast — an exact cosine search for a small corpus, an approximate-nearest-neighbor index (find almost the closest, much faster, for a large one) [8]. Cosine similarity is the closeness score throughout: the cosine of the angle between two text vectors, running 0 (no shared terms, pointing in unrelated directions) to 1 (identical direction, the same wording), so a higher number means more alike.
4. Querying. The new deviation is embedded the same way and the top-k most similar chunks (k is simply how many results you ask for — top-3 here) are returned with their similarity scores — the single most important output for governance, because a low score is a visible signal that the corpus has no good match and the generator should abstain rather than stretch.
5. Generation, constrained. The retrieved chunks are stuffed into the prompt with an instruction to answer only from them and to cite each. Many systems add a reranker between steps 4 and 5 (a second, sharper model that reorders the top-k for relevance) and a final groundedness / faithfulness check that compares each generated sentence against the cited chunk.

The contract is evaluable at every layer, which is the whole reason a regulator can be shown it works. The retriever is measured the way any information-retrieval system is: recall@k (does the right prior case appear in the top k?), precision@k (how many of the k are actually relevant?), and mean reciprocal rank (how high does the first good hit land? — score 1 divided by its rank, so a first-place hit scores 1, second-place 0.5, third 0.33, then averaged across queries) — all checkable against a labeled set of query/relevant-document pairs that a quality reviewer can eyeball. The generator is measured for faithfulness (is every claim supported by a retrieved chunk?) and answer relevance, increasingly with an LLM-as-judge — a second LLM prompted to grade the first one's answer against its cited sources, which scales to far more answers than a human grader could read but inherits the same fluent-and-wrong failure mode it is checking for, so it is itself an imperfect, sampled measurement, which is exactly why the human-review gate stays. The lesson for a regulated plant is that the retriever gives you hard, classical metrics you can validate, while the generator gives you softer ones you can monitor but not fully pin down — so you constrain the generator with the retriever and gate the whole thing behind a signature.

The two NLP tasks under every copilot

Strip the chat interface away and a GMP "copilot" reduces to two well-understood NLP problems, both of which predate LLMs and both of which you can validate.

Triage (classification). A new deviation arrives as free text. Before any drafting, the system should route it: which category (temperature excursion, contamination, out-of-specification assay, equipment fault, documentation), which severity, which owning function. Severity in particular is not an ad-hoc label but a regulated classification — minor / major / critical, assigned under the risk-based thinking of ICH Q9 (the international Quality Risk Management guideline from the International Council for Harmonisation) in the firm's own controlled taxonomy (a fixed, approved list of category names, so everyone classifies against the same defined options) — which is what makes the triage step auditable: the classifier proposes a class that already maps to a defined investigation and escalation path, and the reviewer confirms or overrides it against a written procedure rather than a gut call. This is text classification — vectorize the narrative, predict a label — and it is the step that lets a quality unit prioritize hundreds of deviations a month. The Merck & Co. study is, at its core, a rigorous evaluation of exactly this extract-and-classify capability on real deviation text [1].

Retrieval (similarity search). Given the new deviation, find the most similar historical ones and their CAPAs. This is the retriever step of RAG, and on its own it is enormously useful: an investigator grounded in three real prior cases writes a faster, better investigation than one starting from a blank page. Crucially, you can do this with classical text similarity (TF-IDF and cosine distance) or with neural embeddings — the lexical-versus-dense choice from the embedding step above, with the pipeline unchanged either way — and the classical version runs with no model download, which makes it the honest teaching version.

The reason to name these two tasks explicitly is governance: they are inspectable. A triage classifier's accuracy can be measured against a labeled test set (precision, recall, F1, per class); a retriever's neighbors can be eye-checked for relevance and scored with recall@k. The generative LLM that drafts the final summary is far harder to validate — which is precisely why, in a defensible architecture, it is constrained by these two validatable steps rather than running free.

The GenAI wave sorted by risk: retrieval and summarization at the base (grounded by RAG), the inspectable triage and extraction NLP beneath the copilot, governed drafting of regulated content in the middle — all human-reviewed-and-signed — and agentic autonomy held below a hard regulatory boundary that draft Annex 22 draws and the Purolea warning letter enforces. Original diagram by the authors, created with AI assistance.

A runnable model: deviation_triage.py

The example module examples/platform/ml/deviation_triage.py builds both NLP tasks transparently, with no network and no model download, over a small synthetic corpus of deviation narratives written in the running example's vocabulary — the day-7 fed-batch temperature excursion, the BATCH-2026-004 host-cell-protein (HCP — a residual process impurity, the leftover protein from the production cells that must be cleared to a safe trace; fully defined in QC and release) out-of-specification from hplc_results.csv (the combined release-panel data file; HCP itself is measured by ELISA, an antibody-based assay, matching the module and QC and release), environmental-monitoring excursions, equipment faults, and documentation deviations. The corpus is explicitly synthetic (a real deviation log is confidential), so the classifier metrics are illustrative; the shape of the pipeline is the lesson. One simplification to flag: the five categories fold a true out-of-specification result (a spec breach) and an out-of-trend continued-process-verification (CPV — the ongoing monitoring of an already-validated process) signal — a statistical-process-control (SPC) alert that is still within spec — into the single oos_assay label, where a real triage taxonomy would separate them — they trigger different procedures. Collapsing that distinction here is deliberate teaching noise, not a claim about how a production taxonomy should be drawn.

One thing the synthetic corpus quietly elides is where a real deviation narrative comes from, and it is worth naming because it is the difference between a toy and a governed training set. In a real plant the free text is not free-floating: a deviation record lives inside an MES batch record whose structure follows the ISA-95 enterprise-control hierarchy and serializes in B2MML (Business to Manufacturing Markup Language, the XML schema that carries ISA-95 production records between systems), and the sensor context it references — the day-7 BR101.Temp.PV excursion — arrives over OPC UA (the standard machine-to-MES protocol) carrying the units and equipment identity the narrative names in prose. The corpus this module triages is, in production, the data shadow Book 2's the-data-shadow chapter inventories — the same EBR, alarms-and-events log, and material genealogy — and the metadata those standards attach (which batch, which vessel, which phase, which controlled vocabulary the severity is drawn from) is exactly what makes a triage label auditable rather than a guess. A classifier trained on text stripped of that ISA-95 context is the brittle version; one whose features are anchored to the standardized batch record, and whose corpus is data-governed (data-governance defines who owns and curates it), is the version a quality unit can actually defend. The teaching corpus drops this scaffolding for clarity; a deployment must not.

The triage half vectorizes each narrative with TF-IDF — single words (unigrams) and adjacent word pairs (bigrams) as features, term-frequency counts dampened with a sublinear (log) scale, and the most common English filler words ("the," "of") dropped as low-signal stop-words — and fits a logistic-regression classifier, a standard method that learns a weight per feature and turns their sum into a class probability. A classifier is only trusted on data it was not trained on, so you hold back a fraction as a test set and score predictions there; that hold-back is the train/test split. Note the split: this book has been insistent everywhere else that you must group by batch so that hourly rows from one run cannot leak across the train/test line (training on rows that secretly share a batch with the test rows flatters the score — data leakage). Here the unit of analysis is different — each row is one independent deviation narrative, not a timepoint inside a shared batch — so there is no batch grouping to honor and an ordinary stratified hold-out is the honest choice. We stratify on the label so all five categories appear on both sides of the split, which on a 34-row corpus is the only way the per-class table is even computable. The vectorizer and classifier are kept deliberately small and inspectable so every feature and coefficient is readable:

# examples/platform/ml/deviation_triage.py (excerpt)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, f1_score

def train_triage(seed: int = 2026):
    texts, labels = build_corpus()
    vec = TfidfVectorizer(ngram_range=(1, 2), min_df=1, sublinear_tf=True, stop_words="english")
    X = vec.fit_transform(texts)
    Xtr, Xte, ytr, yte, *_ = train_test_split(
        X, labels, range(len(labels)), test_size=0.33, random_state=seed, stratify=labels)
    clf = LogisticRegression(max_iter=1000, C=4.0)
    clf.fit(Xtr, ytr)
    pred = clf.predict(Xte)
    return {"macro_f1": round(float(f1_score(yte, pred, average="macro")), 3),
            "report": classification_report(yte, pred, zero_division=0)}

The retrieval half is the RAG retriever step, shown without an LLM — cosine similarity over the same TF-IDF space returns the most similar prior deviations with their scores, so a weak match can be flagged rather than blindly trusted. This is steps 2 through 4 of the retriever pipeline above (embed, index, query) in a dozen lines:

# examples/platform/ml/deviation_triage.py (excerpt)
from sklearn.metrics.pairwise import cosine_similarity

def retrieve_similar(query: str, k: int = 3):
    """The retrieval in Retrieval-Augmented Generation, shown without an LLM."""
    texts, labels = build_corpus()
    vec = TfidfVectorizer(ngram_range=(1, 2), min_df=1, sublinear_tf=True, stop_words="english")
    corpus_vecs = vec.fit_transform(texts)
    sims = cosine_similarity(vec.transform([query]), corpus_vecs).ravel()
    order = np.argsort(sims)[::-1][:k]
    return [(texts[i], labels[i], round(float(sims[i]), 3)) for i in order]

Running python platform/ml/deviation_triage.py prints the following, verbatim. The retrieval block is the part that matters: a new "temperature fell below setpoint" deviation correctly surfaces the day-7 BATCH-2026-001 excursion as its top match. The triage macro-F1 is deliberately modest — a teaching result, not a tuned one — because a 34-narrative corpus split into 22 train / 12 test is exactly the small-data regime that breaks classifiers, and the per-class breakdown shows it honestly (the oos_assay class scores 0.00 with only two test examples). The closing NOTE is the chapter's whole argument compressed into the program's own output:

deviation triage + CAPA retrieval (the NLP under the GenAI copilot)
  corpus: 34 SYNTHETIC deviation narratives, 5 categories (illustrative)

-- TRIAGE: TF-IDF (759 features) + LogisticRegression --
  split: 22 train / 12 test   macro-F1 = 0.542 (illustrative)
               precision    recall  f1-score   support

contamination       1.00      0.33      0.50         3
documentation       1.00      1.00      1.00         2
    equipment       0.38      1.00      0.55         3
    oos_assay       0.00      0.00      0.00         2
  temperature       1.00      0.50      0.67         2

     accuracy                           0.58        12
    macro avg       0.68      0.57      0.54        12
 weighted avg       0.68      0.58      0.54        12

-- RETRIEVAL: new deviation, find similar prior cases (RAG retriever step) --
  query: Production bioreactor temperature fell below the 36.5 C setpoint for several h...
  sim=0.363  [temperature  ]  Bioreactor BR101 temperature excursion on day 7 of BATCH-2026-001; jacke...
  sim=0.201  [temperature  ]  Temperature deviation in production bioreactor: setpoint 36.5 C, PV dipp...
  sim=0.163  [temperature  ]  Production bioreactor jacket temperature spiked above the upper NOR duri...

NOTE: triage routes and retrieval grounds -- both ADVISORY. Under draft Annex 22 a generative/probabilistic model is excluded from CRITICAL GMP decisions; a human closes the CAPA. The Purolea warning letter (2026-04-02) is the enforcement anchor.

Read this the way a quality lead would. The retrieval is the immediately useful part: the new temperature deviation pulls back the three most similar historical cases, all genuinely temperature-related and all in the running example's plant, with similarity scores attached so a low-confidence match is visible. Note the scores themselves — the top hit sits at 0.363, not 0.9; on a 34-document corpus of short narratives, even the correct neighbor shares only a fraction of its vocabulary with the query, which is precisely why the score is reported rather than hidden. And treat that magnitude as relative, not absolute: a sparse TF-IDF cosine over short text runs low by construction (most dimensions are zero), so 0.363 is not a portable "good match" threshold and 0.5 is not a universal pass line — swap in dense neural embeddings and the same correct neighbor might score 0.85. The governance signal is not the raw number against a fixed cutoff; it is a threshold calibrated on this retriever and corpus (and the gap between the top hit and the rest), so abstention fires when the best available match is weak relative to what this index normally returns. That is the RAG retriever doing its one job — grounding the investigator in real prior cases instead of a blank page, and flagging how strong (or weak) the grounding is.

One honest caveat about what this demo does and does not show: it exhibits the retriever's mechanics — embed, index, query, and return scored neighbors — but it does not measure the retriever, because the synthetic corpus carries no labeled query/relevant-document pairs. The recall@k, precision@k, and mean-reciprocal-rank numbers from the section above are exactly what you would compute to validate this retriever (a leave-one-out sweep: hold each deviation out, query with it, and check whether its true category-mates rank in the top k), and on a real corpus with a curated relevance set that is the acceptance test a reviewer signs off on. Here the goal is the pipeline shape, not the metric.

The triage classifier, by contrast, is a cautionary tale by design: a macro-F1 of 0.54 on a 34-row corpus is what small data actually buys you. Read the headline number carefully — the honest metric is the macro-F1 of 0.54, not the accuracy of 0.58. (F1 is the harmonic mean of precision and recall — a single score from 0 to 1 that is high only when both are high; macro-F1 averages that score across the five classes, weighting each class equally. In the printed table, support is simply how many test examples fell in each class.) Accuracy averages over predictions and so lets the well-populated, easy classes paper over the failures; macro-F1 averages over classes, giving the two-example oos_assay row (which collapses to zero) the same weight as the rest, so it surfaces precisely the rare-class failure that matters most in a regulated setting, where the dangerous deviation is usually the uncommon one. That refusal to let the easy classes hide the hard ones is the whole reason to report macro-F1 over accuracy here. The lesson is not "build a better classifier"; it is that this number is advisory at best, the LLM drafting on top of it is even less validatable, and a human must read everything. That gap between a confident demo and a thin, honest result is the entire subject of the limits section below.

Anatomy of one deviation-investigation record

A deviation under GenAI assistance, like every artifact in this series, is not a bare LLM answer — it is a structured investigation record where the model's contributions are labeled as such, sit beside their grounding evidence, and are gated behind a human signature. Dissect one the way a quality reviewer would, because the structure is the governance.

One AI-assisted deviation record, fully unpacked: the narrative and the AI-suggested triage (advisory), the RAG-retrieved prior cases that ground the draft (each citing a real document), the LLM-drafted summary and CAPA (stamped DRAFT, human-must-review), and the deterministic governance core — the quality-unit reviewer, the e-signature, the audit trail, and the model version and prompt hash that produced the draft. The AI accelerates every field above the line; the signature below it is the critical decision the model is forbidden to make. Original diagram by the authors, created with AI assistance.

Read the card field by field and the chapter's argument is laid out as data structure.

• Header — record_type, linked_batch. Deviation Investigation, DEV-2026-0142, linked to BATCH-2026-001. Plain identity fields; deterministic, human-authored, no AI involvement. They exist so every downstream AI field is anchored to a real, traceable record.
• Core — narrative, ai_category, ai_severity. The human-authored free-text narrative (the day-7 temperature excursion) is the input; the AI-suggested category (temperature) and severity ride alongside it, each tagged AI-advisory and each carrying a confidence. These are the triage classifier's output from the runnable model above — a routing suggestion, never a final classification. The reviewer can override either with one keystroke, and the override is itself logged.
• Retrieval — similar_prior[] with sim_score and capa_ref. The RAG grounding, field by field: the three most similar prior deviations, each with its similarity score (0.363, 0.201, 0.163 — the exact numbers the retriever printed) and the CAPA reference that closed it, and each marked grounded-in-real-document with a source citation a reviewer can open. The scores are first-class fields, not decoration: a retrieval block where the top score were near zero is a visible instruction to distrust the draft built on it.
• Generation — draft_summary, draft_capa, hallucination_check. The LLM's first-draft investigation summary and draft CAPA, both stamped DRAFT and human-must-review, with a hallucination-check note recording whether each drafted claim was traced back to a retrieved chunk. These are the highest-value and lowest-trust fields on the card — the ones that save the most time and that a reviewer must read most adversarially.
• Governance — qa_reviewer, e_signature, review_decision, audit_trail, model_version, prompt_hash. The deterministic, GxP-controlled core. The quality-unit reviewer, the e-signature (21 CFR Part 11 / EU GMP Annex 11 — the long-standing rules for electronic records and signatures, a different annex from the new AI-specific Annex 22 above), the review decision, the audit-trail entry, and — the field that makes the whole thing auditable forever — the exact model_version and prompt_hash (the prompt is the instruction text fed to the LLM; a hash is a short fixed fingerprint computed from that text — identical text always yields the same hash, any change yields a different one) that produced the draft, so "which model, with which prompt, generated this text?" is answerable for the life of the record. Pinning the model version is what lets the system be re-validated on change instead of drifting silently; the prompt hash is what makes a draft reproducible.
• Relationships (violet panel) — lineage. The record derivedFrom the narrative, is grounded-in the prior CAPAs, was drafted-by a pinned model version, reviewed-by the quality unit, and closed-by a human signature. This is the provenance graph that links the soft AI fields to their hard governance.

Everything above the signature is acceleration; the signature itself is the critical decision the model may not make — and the line between the two runs exactly where the field tags switch from AI-advisory / DRAFT to GxP-controlled.

The card is a graph: typing the record so a copilot can be grounded, not trusted

The violet relationships panel is not decoration — it is the seam where this chapter meets the companion ontology volume, and naming the formal layer explicitly is what turns the card from a tidy diagram into something a regulator and a retriever can both stand on. Those five edges — derivedFrom, grounded-in, drafted-by, reviewed-by, closed-by — are exactly the kind of typed object property an OWL (Web Ontology Language) ontology declares, and the provenance among them is the standard PROV-O pattern (an entity — the record — wasDerivedFrom another entity, wasGeneratedBy an activity — the LLM draft — wasAttributedTo an agent — the pinned model version, and signed off by a human agent). Modeled this way, the record is not a row in a table whose meaning lives in a column name a refactor can break; it is a set of subject-predicate-object triples (RDF, the graph data model) whose meaning is carried by the edge itself. The Book 4 relations-and-genealogy chapter builds bp:derivedFrom as a single owl:TransitiveProperty precisely so a lineage walk reaches any depth from one stated parent edge — which is why the linked_batch field on this card is not a string but a handle into the same genealogy spine: the deviation derivedFrom BATCH-2026-001 plugs straight into the eleven-ancestor lineage back to WCB-CHO-001.

Two consequences make this more than vocabulary. First, a copilot grounded against such a graph is doing GraphRAG — retrieval that follows typed edges rather than loose text — so "what other lots share this deviation's fate?" is answered by traversing derivedFrom to a shared cell bank, not by hoping a paragraph mentions both lots; the ontologies-and-AI chapter works exactly that GraphRAG traversal. A semantically-grounded feature — a value pulled by its ontology IRI and unit, not by a fragile column header — is also what keeps the triage features from silently drifting when an upstream system renames a tag, the failure mode Book 2's semantic-interoperability chapter names. Second, before any subgraph is handed to the generator it can be SHACL-validated (the Shapes Constraint Language, which checks that a graph carries every required field, typed and in range, before it is trusted) — the same closed-world release gate that refuses a non-conformant release doubles as an admission gate that refuses a non-conformant retrieval: a subgraph missing its capa_ref or its approvedBy signer is caught before the LLM fluently fills the hole. And the BFO (Basic Formal Ontology) continuant/occurrent split the same book rests on keeps the card honest at the type level — the deviation record and the temperature excursion it describes are different kinds of thing (a persisting information artifact versus a process that happened and is over), so the model cannot quietly conflate the measurement with the run it came from. The lesson the ML book keeps arriving at, stated in the ontology's own terms: a fluent model is only ever as trustworthy as the typed, shape-validated, FAIR (Findable, Accessible, Interoperable, Reusable) graph it is grounded against — the ontology is the ground truth the model stands on.

The hard limits: hallucination, validation, and leakage

Three limits separate the demo from the deployment, and all three are structural — they do not go away with a bigger model.

Hallucination. An LLM generates fluent, confident text whether or not it is correct; it has no internal sense of "I do not know." In a deviation summary this is dangerous precisely because the output is plausible — a fabricated root cause reads exactly like a real one. The Merck & Co. study names this directly as the "interplay between apparent reasoning and hallucination," and it is why every defensible deployment is RAG-grounded (answer only from retrieved real documents) and human-reviewed [1]. RAG reduces the rate; it does not reach zero, because a model can still misread or over-extend a real passage. The mitigation stack is layered, not single-point: retrieval grounding, then the faithfulness check that traces each sentence to a source, then the hallucination_check field on the record, then the human reviewer — the ISPE GAMP AI Guide's "seven control layers for LLMs" — ISPE is the International Society for Pharmaceutical Engineering, an industry body (not a regulator), and GAMP (Good Automated Manufacturing Practice) is its long-running software-validation framework — is exactly this defense-in-depth posture written as a control framework [16]. One subtle way this stack lies to you is evaluation leakage in the retrieval index itself: if the corpus contains near-duplicate documents — three copies of the same SOP revision, a deviation pasted verbatim into a CAPA — then recall@k and the faithfulness score read artificially high, because the retriever can always find a near-exact twin of any query and the generator can copy it word-for-word and still pass the groundedness check. The metrics look great and the system has learned nothing transferable. The fix is mundane and essential: de-duplicate the index, and validate the retriever against a clean held-out relevance set whose documents do not also sit in the index, so the numbers measure real retrieval rather than memorized duplication.

Validating a non-deterministic model under GxP. Classical computerized-system validation assumes the same input yields the same output, which an LLM does not guarantee (sampling temperature — a randomness dial on the text generator, not the bioreactor temperature, where higher means more varied wording — top-k sampling — drawing the next word from only the k most likely options — and silent model updates all move the output). The draft Annex 22, the ISPE GAMP AI Guide (July 2025, with its seven control layers), and the FDA's risk-based credibility framework all converge on the same posture: a model used for a GMP-relevant purpose must be locked at validation, governed by a predetermined change-control plan, monitored for drift, and held to a level of scrutiny proportional to how much it influences a decision — the FDA framework keys that scrutiny explicitly to the model's "context of use" (a term of art: exactly what the model is used for and how much its output influences a decision — the more it influences, the more scrutiny it draws) [5][16][17]. A continuously-updating cloud LLM is the hardest possible case for this, which is one reason the highest-risk uses are the ones the regulators carve out entirely.

Data leakage and confidentiality. Batch records, deviations, and CMC documents are among a manufacturer's most sensitive data. Sending them to a third-party LLM API risks exposure; training or fine-tuning a shared model on them (further-training a model on your own data so it learns from it) risks the data resurfacing in another customer's output. This is why GMP-credible deployments run on tenant-isolated (your data walled off from other customers) or on-premises (running on the firm's own servers, not a shared cloud) models (e.g., Veeva's agents on Bedrock with customer-isolated or customer-provided models, private deployments behind the firm's own access controls) and why RAG — which keeps the corpus in the firm's control and merely retrieves into a prompt, so the model never absorbs the data — is preferred over fine-tuning on proprietary text [12][8]. The document-level permissions in the retriever do double duty here: they keep a model from surfacing text a given user is not cleared to see, so confidentiality is enforced at retrieval time rather than hoped for at generation time. And the leakage limit has a second face that lands harder in a GMP plant than confidentiality alone: regulated-records integrity. The deviations, batch records, and CMC sections the model reads and drafts against are themselves controlled records under 21 CFR Part 11 and EU GMP Annex 11, held to ALCOA+ (attributable, legible, contemporaneous, original, accurate — plus complete, consistent, enduring, available). An LLM that silently rewrites a retrieved passage, drops a qualifier, or fabricates a value is not just a confidentiality risk; it is a data-integrity event against a record a regulator can subpoena — which is exactly why the drafted fields are stamped DRAFT, the original retrieved chunks are preserved verbatim beside them, and nothing the model touched becomes a record of truth until a human reviews it and the audit trail captures who changed what.

The named products, by maturity and evidence tier

The vendor list under this wave is long and the claims run hot, so it is worth pinning each name to a maturity and an evidence tier — the discipline the vendor-landscape chapter and the case-study ledger apply across the book. None of the customer-outcome numbers below clears the top evidence floor (peer-reviewed-independent); they are vendor- or self-reported unless stated otherwise.

• Aizon — GxP manufacturing-intelligence platform; the intelligent batch record (iBR) enables review-by-exception (a human reads only the records that flag a problem), sitting on contextualized manufacturing data and predictive ML. Maturity: production for the iBR/analytics, pre-announced for the agentic "Agentic Studio" capabilities (conversational generation of batch-release cockpits, OEE (Overall Equipment Effectiveness — a standard plant-productivity metric) trackers, PQR templates, slated for early 2026). Evidence: vendor-self-reported, with the notable peer-reviewed-self-authored exception of Aizon authors' QbD-based (Quality by Design — building quality in by understanding the process up front) AI-qualification procedure (qualification in the GMP sense: documented formal proof that a system performs as intended) [10][14].
• ValGenesis — validation lifecycle management with AI-assisted protocol/validation drafting and digital tech transfer ("Smart GxP"). Maturity: production platform, AI features newer. Evidence: vendor-self-reported (the "80% faster validation" framing is a vendor figure) [18].
• Veeva — Vault AI Agents built natively into the Vault platform, on LLMs from Anthropic (Claude) and Amazon hosted on Bedrock, with customer-isolated or customer-provided model options. Maturity: roadmap — Quality and Safety agents announced for April 2026. Evidence: vendor-self-reported roadmap [12].
• Mareana — a Batch Release Copilot: a rule engine ingesting LIMS (Laboratory Information Management System — the lab's results database) / ERP (Enterprise Resource Planning — the business and inventory system) / batch-record data, validating parameters against specs and historical trends, surfacing exceptions for a human decision, with a generative assistant for root-cause analysis. Maturity: product / early deployment. Evidence: vendor-self-reported [19].
• Microsoft Copilot — RAG over quality-system documents for deviation drafting, deployed in pharma quality systems and described in a practitioner/professional-society account. Maturity: deployed pilot-to-production for non-critical drafting. Evidence: trade/professional-society source [11].
• Siemens + Capgemini + Sanofi (GenAI in MES) — an MES-acceleration program replacing paper batch records with digital ones, with self-reported review-time and deviation reductions; representative of the integrator play (Accenture, Capgemini, Microsoft) selling horizontal copilots into regulated drafting. Maturity: scaling program. Evidence: vendor/integrator-self-reported [13].

The honest summary the ledger forces: the capabilities are real and the platforms exist, but the headline outcome numbers are uniformly self-reported, the agentic tier is roadmap-or-pre-announcement rather than validated production, and the production-grade deployments cluster in the advisory, human-in-the-loop drafting that this chapter has been describing throughout [2].

The unsolved part: validating a system that does not give the same answer twice

The deepest open problem is not hallucination — RAG and human review are a workable, if imperfect, mitigation for that. It is the collision between the definition of validation and the nature of a generative model.

GMP validation rests on reproducibility: you demonstrate that a system, given a defined input, reliably produces a correct, defined output, and you keep it under change control so that property holds for its whole life. A generative LLM violates this at the root. The same prompt can yield different text on two runs; a silent provider-side model update can shift behavior overnight; and even a "locked" model is locked only until the next version, with no guarantee the new one fails in the same places. The traditional answer — re-validate on change — runs into the validation paradox this book named in the MLOps chapter (MLOps = Machine Learning Operations, the practice of deploying and maintaining models in production): the model that most needs continuous updating is the one hardest to keep validated, and freezing it to stay compliant forfeits the improvement that justified using it.

The field has no settled resolution, only a direction of travel. The emerging consensus is to validate the system and its guardrails rather than the model's every output: validate that the retriever returns relevant, permissioned documents (measured with recall@k against a labeled set); validate that the generator is constrained to answer from them and to abstain when they do not support an answer (measured with a faithfulness check); validate the human-review gate and the audit trail; and treat the model itself as a controlled component with a pinned version, a predetermined change-control plan, and ongoing drift monitoring [16][6]. This is a genuine shift in what is under validation — the wrapper, not the weights — and it leans on exactly the inspectable layers (retriever metrics, groundedness checks, the model_version/prompt_hash fields on the record) that the rest of this chapter has been building toward. It is an honest, partial answer. It keeps the human in the loop precisely because no one yet knows how to make a non-deterministic generator deterministic enough to trust unattended — and the draft regulation, by excluding generative AI from critical decisions, is essentially codifying that uncertainty into law.

What this chapter adds to the model suite

This chapter contributes examples/platform/ml/deviation_triage.py to the Book 5 example suite: a standalone, network-free module that demonstrates the two inspectable NLP tasks underneath every GMP copilot. It builds a small synthetic corpus of deviation narratives in the running example's vocabulary, then (1) trains a TF-IDF + logistic-regression triage classifier that routes a deviation into one of five categories, and (2) implements the RAG retriever step as cosine similarity over the same TF-IDF space, returning the most similar prior deviations with scores. It coordinates with — and deliberately does not duplicate — the QC-and-release OOS work (which predicts the numeric OOS) and the packaging anomaly module (serialization_anomaly.py, which reasons over structured events); this module is the text layer. The retrieval output is verbatim and genuinely useful (the temperature query surfaces the real day-7 excursion); the triage metrics are clearly labeled illustrative, and their modesty on a 34-row corpus is the intended lesson about small-data text classification — and about why the generative layer on top must stay advisory.

Why it matters

The paper trail is the part of biomanufacturing that scales worst and costs most in human time: deviations pile up, investigations drag, CMC documents take weeks, and every one is read and signed by a person. Generative AI is the first tool that can genuinely compress that work — drafting, retrieving, triaging — and the time savings reported across the wave, even discounting the self-reported headline numbers, are real enough to be transformative for quality operations. But the same fluency that makes it useful makes it dangerous: a confident, wrong investigation summary is worse than a slow, right one, and a model that decides rather than drafts is the failure mode the FDA has now sanctioned in writing. Getting this layer right means embracing the acceleration while holding the line that the regulators have drawn — the human reviews, the human signs, the model never makes the critical call. That discipline is not a brake on the technology; it is the only thing that lets a manufacturer deploy it at all.

In the real world

The deviation/CAPA and knowledge-management use is the most active corner of the entire GenAI wave, and it is mostly (pilot) edging toward production for non-critical documentation. The peer-reviewed anchor is the Merck & Co. study of GPT-3.5, GPT-4 and Claude-2 on manufacturing deviations — accurate extraction, an explicit hallucination caveat, and a human-in-the-loop hedge [1]. MSD's AWS-Bedrock deviation assistant (with Amazon OpenSearch as the vector store) is publicly described as exploratory RAG [8]; Microsoft Copilot is deployed in pharma quality systems for deviation drafting [11]; McKinsey reports a manufacturer synthesizing ~70% of deviations with first-draft CAPAs for over 80% of cases (consultancy-reported) [7]. On the vendor side, the named-products table above (Aizon, ValGenesis, Veeva, Mareana, and the Siemens/Sanofi/Capgemini GenAI-in-MES collaboration) covers who is selling into this space; the one reading to carry away is that every customer-outcome number there is vendor- or self-reported — Sanofi's ~8x-faster product-quality-review figure included, which should be read as a target rather than a verified result [9].

The governance frame is now concrete, not speculative. The ISPE GAMP AI Guide (July 2025) and its "seven control layers for LLMs" give a validation playbook; the FDA's risk-based credibility framework scales scrutiny to model influence; and draft Annex 22 (consultation July-October 2025) is the first manufacturing-specific AI rule, permitting only static, deterministic models (deterministic = the same input always produces the same output; static = the model does not keep learning after it is locked — the opposite of a generative model that varies its wording and a continuously-learning one that drifts) for critical GMP and excluding generative/continuously-learning AI from critical use [16][17][5][6]. The annex is a draft — finalization is expected mid-2026 and the exclusion is provisional — but its direction is unambiguous. And the enforcement anchor is the Purolea cGMP warning letter (2 April 2026), the FDA's first to cite AI: a firm used AI agents to generate specifications, SOPs, and master production records without quality-unit review required under 21 CFR 211.22(c), the exact opposite of the human-in-the-loop discipline this chapter argues for [3][4][15]. Read together, the message from regulators, the peer-reviewed literature, and the one enforcement action so far is identical: generative AI drafts and retrieves; a qualified human decides and signs.

Key terms

• GxP / GMP / cGMP — GxP is the umbrella of "Good x Practice" regulations a medicine is made under; GMP (Good Manufacturing Practice) is the manufacturing member; cGMP is the FDA's "current" GMP — the same standard kept up to date.
• Large language model (LLM) — a neural model trained on vast text that reads and generates fluent natural language; the engine of the 2023-2026 GenAI wave.
• Generative AI — models that produce new content (text, here) rather than only classifying or scoring; excluded from critical GMP decisions under draft Annex 22.
• Retrieval-Augmented Generation (RAG) — pairing a retriever over the firm's own validated documents with a generator instructed to answer only from those documents, with citations; the architecture that makes LLMs usable in GMP.
• Retriever — the component that finds the most relevant passages in a corpus by vector/text similarity; built as chunk, embed, index, query, and shown here as TF-IDF + cosine similarity, an inspectable, validatable step.
• Embedding — a vector representation of a chunk of text that places semantically similar text nearby; sparse/lexical (TF-IDF, inspectable) or dense/neural (a sentence-transformer, opaque but paraphrase-aware).
• Cosine similarity — the closeness score between two text vectors: the cosine of the angle between them, running 0 (no shared terms) to 1 (identical direction); a higher number means more alike, and the threshold is calibrated per retriever and corpus.
• recall@k / precision@k — the classical retriever metrics: whether the right document appears in the top k results (k = how many results you ask for), and how many of the k are relevant; the hard numbers you validate a retriever with.
• F1 / macro-F1 — F1 is the harmonic mean of precision and recall, a single 0-to-1 score high only when both are; macro-F1 averages it across classes equally, so a rare class cannot hide behind the easy ones.
• Faithfulness (groundedness) — the check that every claim a generator makes is supported by a retrieved source passage; the softer metric you monitor the generator with.
• Hallucination — confident, fluent output that is factually wrong; the central risk of generative AI in a regulated setting, reduced (not eliminated) by RAG grounding and human review.
• Deviation / CAPA — a recorded departure from a procedure (deviation) and the corrective and preventive action that resolves it; the free-text records LLMs are most used to triage and draft.
• Triage (text classification) — routing a free-text deviation into a category and severity to prioritize it; an inspectable NLP task beneath the copilot.
• CMC drafting — using GenAI to draft chemistry, manufacturing, and controls regulatory content; a governed, draft-and-review use.
• Human-in-the-loop — the discipline (and now regulatory expectation) that a qualified person reviews and signs AI output; the model never makes the critical decision.
• Agentic AI — systems that plan and take multi-step actions with minimal human input; confined to non-critical, human-in-the-loop tasks in GMP.
• Draft Annex 22 — the draft EU/PIC/S GMP annex on AI that permits only static, deterministic models for critical GMP and excludes generative/continuously-learning AI from critical use.
• Purolea warning letter — the FDA's first AI-citing cGMP warning letter (2 April 2026), against a firm that used AI to generate GMP records without quality-unit review; the enforcement anchor.
• RDF / OWL / PROV-O — RDF is the subject-predicate-object graph data model; OWL is the logic layer that declares typed properties (a transitive derivedFrom); PROV-O is the standard provenance pattern (entity wasGeneratedBy activity wasAttributedTo agent) that the record's derivedFrom / drafted-by / closed-by edges follow.
• GraphRAG — retrieval-augmented generation whose store is a knowledge graph, so the copilot follows typed edges (traversing derivedFrom to a shared cell bank) rather than loose text; the form a semantically-grounded deviation copilot takes.
• SHACL admission gate — running the same closed-world release shape over a retrieval subgraph before it reaches the generator, so a subgraph missing a required field (a capa_ref, a signer) is refused before the LLM fluently fills the hole; completeness, not correctness.
• ISA-95 / B2MML / OPC UA — the manufacturing-data standards under a real deviation: ISA-95 structures the MES batch record, B2MML serializes it between systems, and OPC UA carries the sensor context with units and equipment identity — the metadata that makes a triage label auditable rather than a guess.

Where this leads

The manufacturing spine is complete — every step from discovery to distribution learned, the whole-system layers (hybrid twins, MLOps, operations, and now language) mapped. Part VII turns from how ML works in biomanufacturing to who sells it and what is actually real. The next chapter, The Vendor Landscape: Who Sells What, and What Is Real, takes the names that have recurred throughout this book — Sartorius, AspenTech, Aizon, DataHow, Insilico/Yokogawa, Cytiva, Körber, ValGenesis, Veeva — and sorts their claims into production, pilot, and press release, so the buyer can tell the validated capability from the marketing.