The Reference Architecture: One Stack, Layer by Layer

📍 Where we are: the blueprint. Before we type a single command, we unfold the whole platform onto one page — every layer, every open-source tool, every ISA-95 level — and draw the line where open source stops and commercial systems begin.

The preface made you a promise: clone one repository and build a working bioprocess data platform, layer by layer, in open source. This chapter is the map of that platform. Think of it as the architectural drawing you pin to the wall before construction starts — the one every later chapter quietly points back to.

We will not run anything yet. Chapter 2 does that. Here we do the harder thing: we make sure you can see the shape of the whole stack — how a number born on a pH probe travels through a message bus, lands in a historian, gets stitched to a batch record, becomes a triple in a knowledge graph, and finally helps assemble a regulatory submission — before you build the first piece of it.

The simple version

Imagine a building with eight floors. The ground floor is the loading dock where raw goods (sensor readings) arrive. Each floor above adds meaning: one labels and routes the deliveries, one warehouses them, one cross-references every box against the order it belongs to, one lets inspectors trace any box back to its origin. The top floors are where the regulators visit. Our job in this chapter is to draw the floor plan, name the open-source tool that staffs each floor, and mark — in red ink — the floors where the free staff cannot pass inspection alone and you must hire a licensed professional.

What this chapter covers

• The layered blueprint, top to bottom, and why the companion repo's build order follows it.
• Each layer mapped to an ISA-95 / Purdue level and to the open-source tool we chose for it.
• Where the honest OSS↔commercial boundary falls, and why it falls there.
• The one design decision that makes the whole stack joinable: the historian and the batch model living in the same database.
• A component-and-license inventory, and the data-integrity question that recurs through every chapter.

The blueprint on one page

Every layer of the platform has the same job description: take the data from the layer below, add one kind of meaning, and hand it up. Read from the bottom:

Layer	What it adds	ISA-95 / Purdue level	OSS tool in this book
Edge connectivity	a standard, self-describing way to read sensors	Level 0–2 (sensors, control)	OPC UA (asyncua)
Message bus	a named, real-time stream of every value	Level 2–3 boundary	MQTT + Sparkplug B (Mosquitto)
Historian / TSDB	durable, queryable time-series at scale	Level 3	TimescaleDB hypertable
Batch & equipment model	the context that makes a number mean something	Level 3	PostgreSQL (ISA-88/95)
Contextualization	the join: this value, this batch, this phase	Level 3	SQL views
Semantics	machine-traceable lineage across systems	Level 3–4	RDF / SPARQL (Apache Jena Fuseki)
Compliance / trust	the audit-trailed record of truth	Level 3–4	Postgres audit + hash chain
Analytics	prediction and process understanding	Level 3–4	Python (SPC, PLS soft-sensor)

The mapping to ISA-95 — the international standard (IEC 62264) for integrating enterprise and control systems, refreshed in its 2025 edition — is not decoration [1]. It tells you where each tool legitimately lives and, crucially, where the boundaries between them sit. The single most important boundary in this whole table is the one between Level 2 (the control system that actually runs the bioreactor) and Level 3 (everything we build in this book). The validated control system is sacred; we never write into it. We read from it.

That principle has a name. NAMUR — the European user association of automation technology — published the NAMUR Open Architecture (NOA) concept precisely for this: a second, read-mostly data channel that feeds monitoring, historization, and optimization without altering the validated core process control system [2]. Almost everything in this book lives on the NOA side of that line. When you hear us say "we never touch the DCS," NOA is the standard that blesses it.

The reference architecture, bottom to top. Data flows upward: each layer adds one kind of meaning and hands it to the next. The unshaded layers are pure open source you build and run on a laptop; the shaded compliance band near the submission is the honest hybrid — the GxP last mile of validation, qualified signatures, and vendor accountability that open source alone does not deliver. Original diagram by the authors, created with AI assistance.

The same stack, as a dataflow

The table reads bottom-up, the way you build it. The data, of course, flows the other way. Here is the same architecture as the journey of a single pH reading on bioreactor BR101:

Notice that the historian (D) and the batch model (E) sit side by side and are connected by a plain line, not an arrow. That is the keystone of the design, and it deserves its own section.

One database for the historian and the batch model

In most facilities the historian and the batch/relational world are two different products from two different vendors, and joining them is a nightly ETL job that someone babysits. We refuse that split. Look at how the companion repo defines its database, in examples/platform/compose/compose.yaml:

services:
  # --- core --------------------------------------------------------------
  postgres:
    # timescale/timescaledb IS PostgreSQL + TimescaleDB, so the historian
    # hypertable and the ISA-88/95 batch model live in one joinable database.
    image: timescale/timescaledb:2.17.2-pg17
    profiles: ["core"]
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-bioproc}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-bioproc}
      POSTGRES_DB: ${POSTGRES_DB:-bioproc}
    ports: ["5432:5432"]
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ../db:/docker-entrypoint-initdb.d:ro   # 00-60 schema files run on first init

TimescaleDB is not a separate database — it is a PostgreSQL extension. Its core abstraction, the hypertable, is an ordinary PostgreSQL table that is automatically partitioned by time into chunks, so high-rate sensor data behaves like any other relational table while staying fast at scale [3]. Because the historian is PostgreSQL, the time-series and the batch context are not two systems pretending to talk — they are two schemas in one database you can join with plain SQL. The historian lives in schema ts; the batch model lives in schema s88. Here is the hypertable, from examples/platform/db/20-historian.sql:

CREATE TABLE ts.sensor_reading (
    ts       timestamptz      NOT NULL,
    tag      text             NOT NULL,
    value    double precision,
    unit     text,
    quality  smallint         NOT NULL DEFAULT 192,  -- OPC UA: 192 Good, 64 Uncertain, 0 Bad
    batch_id text
);

SELECT create_hypertable('ts.sensor_reading', 'ts', chunk_time_interval => INTERVAL '1 day');

That quality column is small but load-bearing. OPC UA — the platform-independent, service-oriented protocol that carries data and its metadata from the sensor up through Level 3 [4] — attaches a status code to every value: 192 for Good, 64 for Uncertain, 0 for Bad. We preserve it all the way into the historian. When an inspector later asks whether a value was trustworthy at the moment it was recorded, the answer is a column, not a guess. This is the ALCOA+ attribute "Original" made concrete: the data integrity guidance that frames the rest of this book expects the original measurement and its context to survive intact [5].

The layer that turns numbers into knowledge

A row in ts.sensor_reading says BR101.DO.PV = 41.3 %sat at 2026-03-08T11:00:00Z. True, but nearly useless on its own. Was the batch growing or producing? Was this DO reading inside the controlled band for that phase? Answering needs the batch model. The contextualization layer is the join that supplies it, defined once in examples/platform/db/60-views.sql:

-- A reading with its full batch + phase context.
CREATE OR REPLACE VIEW s88.v_batch_sensor AS
SELECT r.ts, r.tag, r.value, r.unit, r.quality, r.batch_id,
       b.product_id, b.recipe_id, b.unit_id,
       bp.phase_id, ph.name AS phase_name
FROM ts.sensor_reading r
JOIN s88.batch b              ON b.batch_id = r.batch_id
LEFT JOIN s88.batch_phase bp  ON bp.batch_id = r.batch_id
     AND r.ts >= bp.start_ts AND (bp.end_ts IS NULL OR r.ts < bp.end_ts)
LEFT JOIN s88.phase ph        ON ph.phase_id = bp.phase_id;

This single view is the architectural payoff of keeping everything in one database. It joins a time-series reading (ts.sensor_reading) to the batch it belonged to (s88.batch) and to the ISA-88 phase that was active at that instant (s88.batch_phase → s88.phase). The batch model itself follows the ISA-88 / ISA-95 hierarchy — enterprise → site → area → unit, and recipe → operation → phase — modeled in examples/platform/db/10-isa88-95.sql. The repo seeds it with a concrete fed-batch CHO line in examples/platform/db/seed/seed_cho_line.sql:

INSERT INTO s88.unit VALUES
    ('BR101',         'UPSTREAM',   'Production Bioreactor 101', 'bioreactor',     'Sartorius', 'Biostat STR 50'),
    ('N1SEED',        'UPSTREAM',   'N-1 Seed Bioreactor',       'bioreactor',     'Sartorius', 'Biostat STR 10'),
    ('PA01',          'DOWNSTREAM', 'Protein A Capture Skid',    'chromatography', 'Cytiva',    'AKTA process'),
    ('TFF01',         'DOWNSTREAM', 'UF/DF Skid',                'tff',            'Cytiva',    'AKTA flux'),
    ('FILL-LINE-01',  'FILL',       'Aseptic Fill Line',         'fill_line',      'Bausch+Stroebel', 'KSF');

That is our running case in data: one production bioreactor, an N-1 seed, a Protein A capture skid, a UF/DF skid, and a fill line — the fed-batch CHO + Protein A monoclonal-antibody process the whole trilogy follows. (A perfusion / multi-column continuous variant of the same line appears as a sidebar in later chapters; the schema is deliberately identical so the continuous case reuses every table.) Once the contextualization view exists, a question that used to be an archaeology project — "what was dissolved oxygen doing, by phase, for the golden batch?" — becomes the one-line query make contextualize runs against s88.v_batch_sensor.

The message bus: a named stream, not a tangle of point-to-point links

Between the sensors and the historian sits the message bus. Its job is to turn a chaos of point-to-point connections into one named, real-time stream that any consumer can subscribe to. We use MQTT — a lightweight publish/subscribe protocol — carried by the Mosquitto broker, pinned in the same compose file:

  mosquitto:
    image: eclipse-mosquitto:2.0.22
    profiles: ["core"]
    ports: ["1883:1883"]
    volumes:
      - ../mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro

Raw MQTT topics are a free-for-all, though, and a free-for-all is the enemy of data integrity. So on top of MQTT we adopt Sparkplug B, the Eclipse specification that imposes a standardized topic namespace, compact payloads, and — most importantly — birth/death session state so a consumer always knows whether a device is alive and what it is reporting [6]. Sparkplug is the mechanism behind the "Unified Namespace" idea you will meet in Chapter 4: one self-describing, broker-mediated address space for the whole plant. We will build the OPC UA → Sparkplug collector in Chapter 5; here, the point is simply that the bus is a named layer with a standard, not improvised plumbing.

Where open source stops: the honest boundary

Now the red ink. Trace the dataflow upward again and watch where pure open source runs out of road.

Capture, historize, contextualize, visualize, reason — the lower six layers — are pure, runnable open source. You will build all of it on a laptop and make test proves it works. That is roughly the first 80% of the platform.

The compliance and trust band is where the honest hybrid begins. No open-source component is 21 CFR Part 11 compliant out of the box [7] — nor does it satisfy the parallel EU Annex 11 regime — and none will be by download alone. We demonstrate a trust layer — a system-versioned audit trail and a cryptographic hash chain in PostgreSQL (Chapters 20–21) — and it is genuinely useful: it makes tampering detectable. But it does not make tampering impossible; a database superuser can disable the trigger that writes the audit row. Part 11 compliance is a property of a validated system and its procedures, not of any single tool [7]. The modern guidance agrees on the method: GAMP 5's second edition adds a risk-based, critical-thinking approach (and an open-source appendix) that lets OSS be used in GxP inside a validated lifecycle [8], and the FDA's Computer Software Assurance guidance reframes that lifecycle as risk-proportionate assurance — leveraging logs, audit trails, and supplier evidence rather than documentation theater [9]. That is exactly the work we show, and exactly why no tool arrives compliant.

The commercial systems are the other side of the hybrid. AVEVA PI, SAP, Emerson DeltaV, Siemens, and commercial LIMS cannot run on a laptop and are license-locked. The integration code we write is real; its counterpart is a clearly labelled mock with the same API contract and a documented production swap (Chapters 17–19). And the OT/IT boundary the whole architecture straddles is itself a security perimeter, governed by ISA/IEC 62443 — the zones-and-conduits standard (formerly ISA-99) that says where a firewall, a data diode, or a one-way NOA channel belongs between the Purdue levels [10]. The final chapter scores, line by line, exactly what you trade away at each boundary.

A component and license inventory

Because we recommend specific tools, we owe you their licenses — and the 2026 traps. The core stack in examples/platform/compose/compose.yaml is small and deliberate:

Component	Pinned image	License	Note for 2026
PostgreSQL + TimescaleDB	timescale/timescaledb:2.17.2-pg17	PostgreSQL License + Apache-2.0 (OSS build)	We use only Apache-2 features; the TSL columnstore/compression/HA are the trap we avoid.
MQTT broker	eclipse-mosquitto:2.0.22	EPL-2.0 / EDL-1.0	Pinned to the stable 2.0.x line for reproducibility.
Dashboards	grafana/grafana-oss:11.4.0	AGPL-3.0	Local use is fine; redistributing or hosting it as SaaS for others triggers AGPL obligations.
Triplestore	apache/jena-fuseki:5.2.0	Apache-2.0	Verify the digest — the community image moved.
Metrics store	victoriametrics/victoria-metrics:v1.108.1	Apache-2.0	Shipped instead of InfluxDB 3 to dodge the v3 license flip.

Every image is pinned by tag, and in the repo's lock file by digest as well, so the running stack, the license inventory, and the supplier register that Chapter 22 generates for validation can never quietly drift apart. The historian's OSS posture is written right into the schema comment in examples/platform/db/20-historian.sql:

-- Only Apache-2/Community features are used (no TSL columnstore/compression),
-- so the stack stays cleanly open-source — the trade-off Chapter 13 discusses.

That single comment captures the book's whole license philosophy: we use open features deliberately and flag the proprietary ones loudly, so a tool you adopt for being "open" does not ambush you later.

Why it matters

An architecture diagram is not a decoration; it is a decision record. Every later chapter is a thin slice over this one shared blueprint — the companion repo is built exactly as "thin chapters over a thick shared platform." Chapter 5 does not redefine the historian; it writes into the ts.sensor_reading you saw above. Chapter 14 does not invent a join; it extends the s88.v_batch_sensor view. Chapter 20 does not bolt on a new database; it adds an audit schema beside the ones already there. Because the layers are defined once and reused, the build order is the architecture: make up brings up the core (database, broker, dashboards), make seed loads the ISA-88/95 line, and every chapter after that turns on one more profile.

Getting the blueprint right early is also what makes the regulated end achievable at all. If the historian and the batch model were two disconnected products, the contextualization view — and with it every audit trail, lineage query, and golden-batch comparison built on top — would be a fragile ETL pipeline instead of a one-line SQL join. The architecture is the difference between a batch investigation that takes three weeks and one that takes an afternoon.

In the real world

Walk into a modern biomanufacturing facility and you will find this exact layering, even if no one drew it on a wall. There will be OPC UA servers on the skids, a historian (often a commercial one), a relational MES holding the batch model, and — increasingly — open-source tools like Grafana and PostgreSQL running alongside the validated systems. The skill the industry actually pays for is knowing where the boundary between them sits and why it sits there. ISA-95 gives that boundary a vocabulary [1]; ISA/IEC 62443 gives the OT/IT perimeter its controls [10]; NOA gives the read-mostly analytics channel its legitimacy [2].

The institutional momentum is real. NIIMBL — the U.S. public-private National Institute for Innovation in Manufacturing Biopharmaceuticals — exists to accelerate exactly this kind of capability. Its pilot-scale cGMP (current Good Manufacturing Practice) facility with the University of Delaware, SABRE, broke ground in April 2024 and is under construction as of mid-2026. SABRE is a facility, not a data program — but facilities like it are where architectures like this one get exercised against real, regulated production. And in every such plant the same question recurs, the one this book returns to in every chapter: which layer holds the trustworthy, audit-trailed record of truth? The blueprint you just read is our answer to where that record can live; the trust chapters are our honest accounting of how far open source can take it, and where it cannot.

Key terms

• Reference architecture: the layered blueprint of the whole platform, where each layer adds one kind of meaning and maps to an ISA-95 level and an open-source tool.
• ISA-95 (IEC 62264): the standard layered model (Levels 0–4) for integrating enterprise and control systems, used to place each tool.
• Purdue model / ISA-99 → ISA/IEC 62443: the OT/IT layering and its zones-and-conduits security standard, defining where boundary controls belong.
• NAMUR Open Architecture (NOA): the concept of a second, read-mostly data channel for monitoring and optimization that never alters the validated control system.
• OPC UA: the platform-independent, self-describing protocol that carries each value plus its quality status from sensor to application.
• MQTT / Sparkplug B: the lightweight publish/subscribe transport (Mosquitto) and the specification that gives it a standardized namespace and birth/death state — the basis of the Unified Namespace.
• Hypertable: a TimescaleDB table auto-partitioned by time, the historian abstraction that keeps high-rate sensor data inside PostgreSQL.
• Contextualization: the SQL join (s88.v_batch_sensor) that ties a raw reading to its batch, equipment, and ISA-88 phase.
• Honest hybrid: the stance that pure OSS covers ~80% of the stack while the GxP last mile (validation, e-signatures, HA, vendor accountability) is met with hardening or commercial systems.
• Record of truth: the audit-trailed, trustworthy electronic record subject to Part 11 / Annex 11 — the recurring question of which layer holds it.

Where this leads

You now hold the map: eight layers from sensor to submission, each pinned to an ISA-95 level and an open-source tool, with the OSS-versus-commercial boundary drawn in ink. The next chapter, Standing Up the Stack: One docker compose up, turns the map into a running machine. We bring up the core profile — PostgreSQL + TimescaleDB, Mosquitto, and Grafana — with a single pinned command, start the CHO simulator alongside it (a Python package, not a Compose service), prove the stack is alive with a first-data-point smoke test, and watch the first reading flow into the historian you just designed.