Standing Up the Stack: One docker compose up

📍 Where we are: Part I, Chapter 2 — we read the blueprint last chapter; now we boot the whole platform on your laptop so every later chapter has something real to run against.

The simple version

Think of the companion repo as a flat-pack data plant. Every machine — the database, the message broker, the dashboards, the bioreactor simulator — comes in its own sealed box (a container). One instruction sheet (compose.yaml) says which boxes to open, how to wire them together, and how to check each one switched on. You type one command, and the plant assembles itself. Type another, and it folds back into boxes — leaving no mess on your machine.

What this chapter covers

This is the hands-on turning point of the book. By the end you will have cloned one repository and brought a real, multi-service bioprocess data platform to life on a single laptop. We will:

• Walk the one compose.yaml that defines the whole core stack, and explain why each service is there.
• Explain why pinned image tags matter — the influxdb:latest→v3 license trap is the cautionary tale.
• Run the Makefile that is the exact command surface the book prints.
• Confirm the stack is alive with a first-data-point smoke test.
• Meet the deterministic CHO simulator the whole book feeds from.

Everything below comes from files that exist in examples/ and were run. No invented flags, no invented output.

One file, the whole core

A modern container platform lets you describe a set of services declaratively — what image each runs, what ports it exposes, what volumes it mounts, how to tell it is healthy — and then bring them all up with a single command [1]. That description is a formal artifact: the Compose Specification defines the schema for services, networks, and volumes, so the same YAML behaves identically on your machine, a colleague's, and the CI runner [2].

Here is the top of the real file, in examples/platform/compose/compose.yaml:

# compose.yaml — the base stack for "From Sensor to Submission".
# One file defines every service; Docker Compose PROFILES gate what comes up so a
# reader only pays for the chapter they are on:
#   core       Ch 1-4, 13-15  (db + broker + dashboards; the CHO simulator is a
#                              separate Python package run via `make data`)
#   capture    Ch 5-12        (collectors, edge gateway, ingesters)
#   semantics  Ch 16          (triplestore)
#   trust      Ch 20-21       (identity, signing, object store)
#   analytics  Ch 26          (notebooks, model tracking)
# Bring up just the foundation with:  docker compose --profile core up -d
#
# Images are pinned by tag for reproducibility; the matching manifest digests are
# recorded in versions.lock (revisited in the supply-chain chapter, Ch 22).

name: sensor-to-submission

The key design idea is thin chapters over a thick shared platform. Every service in the entire book is declared exactly once, in this single file, and tagged with a Compose profile (core, capture, semantics, trust, analytics). docker compose --profile core up starts only what Chapters 1–4 need — roughly 3 GB of RAM — and each later Part turns on one more profile. You never re-declare the stack; you only ever switch a profile on. Your laptop's memory and CPU scale with the chapter you are actually reading.

The core profile is the always-on foundation. Note one deliberate choice in examples/platform/compose/compose.yaml:

  postgres:
    # timescale/timescaledb IS PostgreSQL + TimescaleDB, so the historian
    # hypertable and the ISA-88/95 batch model live in one joinable database.
    image: timescale/timescaledb:2.17.2-pg17
    profiles: ["core"]
    <<: *restart
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-bioproc}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-bioproc}
      POSTGRES_DB: ${POSTGRES_DB:-bioproc}
    ports: ["5432:5432"]
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ../db:/docker-entrypoint-initdb.d:ro   # 00-60 schema files run on first init
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-bioproc} -d ${POSTGRES_DB:-bioproc}"]
      interval: 5s
      timeout: 5s
      retries: 20

There is no separate "time-series database" container. The timescale/timescaledb image is PostgreSQL [3] with the TimescaleDB extension already installed [4]. That single decision pays off for the rest of the book: the high-rate sensor history and the ISA-88/95 batch model live in the same database, so a query can join "what the dissolved-oxygen probe read at 14:32" to "which batch and recipe phase was running" without copying data between systems. We will lean on that join hard in the contextualization chapters.

Two more details in that block earn their keep. The volumes line mounts ../db into PostgreSQL's first-boot init directory, so the numbered schema files (00-init.sql through 60-views.sql) run automatically the first time the database starts — the schema is code, not a manual step. And healthcheck runs pg_isready every five seconds so the platform can know, not guess, when the database is ready to accept connections. Healthchecks are how later chapters' tests wait for a clean dependency before they run.

The rest of core is the broker, the dashboards, and a triplestore/metrics pair that ride along under other profiles:

  mosquitto:
    image: eclipse-mosquitto:2.0.22
    profiles: ["core"]
    <<: *restart
    ports: ["1883:1883"]
    volumes:
      - ../mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
    healthcheck:
      test: ["CMD-SHELL", "mosquitto_sub -t '$$SYS/#' -C 1 -W 3 -h localhost || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 10

  grafana:
    image: grafana/grafana-oss:11.4.0
    profiles: ["core"]
    <<: *restart
    ports: ["3000:3000"]
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD:-admin}
      GF_USERS_ALLOW_SIGN_UP: "false"
    volumes:
      - ../dashboards/provisioning:/etc/grafana/provisioning:ro
      - grafana:/var/lib/grafana
    depends_on:
      postgres:
        condition: service_healthy

mosquitto is the MQTT broker — the lightweight publish/subscribe message bus that simulator telemetry will flow across in the capture chapters [5]. grafana-oss is the dashboard layer that queries the historian and draws the batch-overlay and golden-batch charts the book builds [6]. Notice grafana declares depends_on with condition: service_healthy against postgres: Grafana will not start drawing until the database has passed its healthcheck. Two more services, fuseki (the knowledge-graph triplestore) and victoriametrics (stack self-monitoring), sit behind the semantics and analytics/ops profiles and stay dormant until those Parts.

Telegraf and Node-RED — the plugin-driven metrics agent [7] and the browser-based low-code flow editor [8] — are not in the core profile because they belong to the edge-gateway chapter; they switch on with the capture/edge profile later, exactly the same way. The point is that the file you are reading now already contains the seats for them.

The one Compose file fans out into a handful of pinned, health-checked containers; the Makefile is the only command surface you ever type, and profiles decide how much of the plant is powered on. Original diagram by the authors, created with AI assistance.

Why pinned tags matter (the latest trap)

Look again at every image: line: timescale/timescaledb:2.17.2-pg17, eclipse-mosquitto:2.0.22, grafana/grafana-oss:11.4.0. None of them says :latest. That is not fussiness; it is the difference between a reproducible plant and a time bomb.

Containers are distributed as OCI images — a content-addressable manifest plus layers, identifiable by an immutable digest [9]. A tag like 2.17.2-pg17 is a friendly label pointing at one such digest; a tag like latest points at whatever the maintainer pushed most recently. Semantic versioning gives the tag meaning: MAJOR.MINOR.PATCH, where a MAJOR bump signals a breaking change [10]. Pin the version and you can reason about what an upgrade will and will not break.

The canonical horror story is InfluxDB. A reader who wrote influxdb:latest in 2024 woke up one morning pulling InfluxDB 3 — a near-total rewrite with a changed storage engine and a changed license posture — silently, in place, with no warning. The book sidesteps that entire class of accident by avoiding InfluxDB (we ship VictoriaMetrics under Apache-2.0 instead) and, more importantly, by pinning everything. compose.yaml pins each image by its human-readable tag; a companion platform/versions.lock records the matching immutable manifest digest (<image:tag> sha256:…) for each one. The supply-chain chapter (Chapter 22) builds on that lockfile to cross-check the running stack, the license inventory, and the supplier register against one pinned list — so they cannot silently drift apart.

A few 2026 license traps are worth flagging as you choose versions, because they bite quietly: TimescaleDB's columnstore/compression and HA features live under the TSL license, so we run the Apache-2.0 OSS build and use only OSS-safe partitioning and retention; Grafana is AGPL-3.0 (perfectly fine to run locally, but redistributing or hosting it as a service for others triggers obligations); InfluxDB v3, EMQX's BSL, and Redpanda's RCL are the other landmines this stack steps around on purpose.

The Makefile is the command surface

You will never type a raw docker compose incantation in this book. Every action goes through make, and the book prints exactly what you type. Here is the real examples/Makefile:

COMPOSE := docker compose -f platform/compose/compose.yaml
PY := sim/.venv/bin/python
export DATABASE_URL ?= postgresql://bioproc:bioproc@localhost:5432/bioproc

.DEFAULT_GOAL := help
.PHONY: help venv up down seed data load contextualize alcoa soft-sensor test clean

help: ## list targets
	@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN{FS=":.*?## "}{printf "  %-14s %s\n", $$1, $$2}'

venv: ## create the Python env and install the simulator (uv)
	cd sim && uv venv --python 3.12 .venv && uv pip install --python .venv -e . "psycopg[binary]" scikit-learn

up: ## bring up the core stack (postgres+timescale, mosquitto, grafana)
	$(COMPOSE) --profile core up -d
	@echo "waiting for postgres..." && sleep 3
	@until docker exec sensor-to-submission-postgres-1 pg_isready -U bioproc >/dev/null 2>&1; do sleep 2; done
	@echo "core stack up."

make help self-documents from the ## comments, so the menu and the code never disagree. make up brings up the core profile and then blocks until pg_isready succeeds, so the command does not return "done" until the database can actually take a connection. That polling loop is the small, honest difference between "the container started" and "the service is ready."

The full build order for the foundation is short:

make venv          # Python env + the simulator (uv)
make data          # generate every dataset deterministically + MANIFEST.sha256
make up            # bring up the core stack (postgres+timescale, mosquitto, grafana)
make seed          # load the ISA-88/95 reference CHO line
make load          # load the datasets into the historian + lab tables

make down stops the stack but keeps your data in named volumes; make clean runs docker compose down -v to delete the volumes when you want a truly fresh start. Because the whole environment is one declarative file plus one command, tearing it down leaves nothing scattered across your machine.

The first data point: a smoke test

Bringing services up is not the same as proving the platform works end to end. The smoke test for this stack is the simplest possible question: can a number land in the historian and come back joined to a batch?

After make up && make seed && make load, the simulator's datasets are in PostgreSQL+TimescaleDB. A first sanity check is to count what landed in the historian hypertable directly:

docker exec -e PGPASSWORD=bioproc sensor-to-submission-postgres-1 \
  psql -U bioproc -d bioproc \
  -c "select tag, count(*), round(min(value)::numeric,2) lo, round(max(value)::numeric,2) hi \
      from ts.sensor_reading where batch_id='BATCH-2026-001' group by tag order by tag limit 4;"

      tag      | count |  lo   |  hi
---------------+-------+-------+-------
 BR101.DO.PV   | 20160 | 30.04 | 43.77
 BR101.Temp.PV | 20160 | 36.36 | 37.12
 BR101.Titer.PV| 20160 | -0.11 |  5.82
 BR101.pH.PV   | 20160 |  6.91 |  7.08

Those ranges are the fed-batch process telling the truth about itself: temperature held near 37 °C, pH spanning roughly 6.9–7.1, dissolved oxygen riding between about 30 and 44 %sat, and titer climbing from essentially zero (a slightly negative measurement at inoculation, from the soft-sensor noise) to roughly 6 g/L over the run. Each tag has 20,160 rows — one every minute across the 14-day batch (the full-resolution fedbatch_timeseries.parquet that make load ingests; datasets/ also ships a 10-minute CSV downsample for file-replay chapters).

The real smoke test, though, is the join. make contextualize (which we build properly in the contextualization chapter) runs exactly this query against the same stack:

select phase_name, count(*) n, round(avg(value)::numeric,1) avg_DO
  from s88.v_batch_sensor where batch_id='BATCH-2026-001' and tag='BR101.DO.PV'
  group by phase_name order by min(ts);

If that returns dissolved-oxygen averages broken out by recipe phase, the platform is alive in the way that matters: a raw sensor value, captured into the historian, has been reunited with its ISA-88 process context inside one query. That is the whole platform in miniature, and it is the proof the rest of the book builds on.

The CHO simulator the whole book feeds from

The book has no real bioreactor, so it ships a deterministic one. The Python package bioproc_sim (installed by make venv, driven by make data) generates every dataset in the book from one fixed master seed, SIM_SEED=2026, so the 14-day fed-batch trace is byte-for-byte identical on every machine. That determinism is not a gimmick — it is what lets CI assert a MANIFEST.sha256 and catch any silent drift in the data.

The fed-batch run models a CHO culture with logistic viable-cell growth, Monod glucose/glutamine kinetics (lactate is produced as a byproduct during growth and consumed late, not a limiting substrate), antibody titer accumulating with the integral of viable cells (a growth-associated production term), and PID-controlled DO and pH with bounded noise. It even seeds a deliberate 0.5 °C excursion on day 7 and scheduled bolus feeds on days 3, 5, 7, 9, 11, and 13 — so later chapters have real events to detect, alarm on, and review. A row of the golden trace looks like this:

ts,tag,value,unit,quality,batch_id
2026-01-05 00:00:00+00:00,BR101.DO.PV,40.8224,%sat,192,BATCH-2026-001
2026-01-05 00:00:00+00:00,BR101.Temp.PV,37.0145,degC,192,BATCH-2026-001
2026-01-05 00:00:00+00:00,BR101.Titer.PV,-0.0045,g/L,192,BATCH-2026-001
2026-01-05 00:00:00+00:00,BR101.pH.PV,7.0511,pH,192,BATCH-2026-001

The quality column carries the OPC UA StatusCode severity (192 = Good, 64 = Uncertain, 0 = Bad), the unit keeps the engineering unit attached to the number, and batch_id is the thread that ties every reading back to the ISA-88/95 model. The same engine can also stream live to an OPC UA server and to Mosquitto for the capture chapters, or dump flat goldens to datasets/ for chapters you want to follow by replaying files without booting a producer. One seed, one source of truth, every number in the book.

Why it matters

Everything downstream — the historian, the batch model, contextualization, the audit chain, the soft-sensor — assumes a working, reproducible foundation. Get that wrong and every later chapter inherits the flakiness. Get it right and the book becomes something you run, not something you read.

There is a regulatory dividend too. A version-pinned, declarative, automated environment is exactly the artifact a qualification effort wants. GAMP 5 (2nd Edition) frames a risk-based lifecycle for GxP computerized systems and gives explicit attention to infrastructure qualification and to open-source software [11]. When your infrastructure is code — one Compose file, one lockfile of digests (versions.lock), one Makefile — your installation evidence is reproducible and reviewable rather than a screenshot of someone's terminal. The FDA's Computer Software Assurance final guidance points the same way: assurance should be risk-based and least-burdensome, leaning on logs, automation, and supplier evidence rather than ritual documentation [12]. A clean make up that boots a pinned stack and passes a healthcheck is precisely the kind of objective, repeatable evidence those frameworks reward.

In the real world

Real plants do not run a single laptop Compose file, of course. A production historian might be AVEVA PI on dedicated, highly-available hardware; the DCS is Emerson DeltaV or Siemens; the LIMS is commercial and validated. Those systems cannot run on a laptop and are not open source — which is why this book is honest about being a hybrid: the open-source core here gets you perhaps 80% of the way, and the GxP last mile (Part 11 e-signatures, vendor accountability, validated HA) is where commercial systems and formal validation take over. No tool in this stack is Part 11-compliant out of the box, and saying otherwise would be marketing, not engineering.

But the architecture this chapter stands up is the same one the big shops use, just at laptop scale: a time-series historian beside a relational system of record, a message bus between the floor and IT, dashboards over the top. NIIMBL — the U.S. public-private Institute for Innovation in Biopharmaceutical Manufacturing — and its SABRE facility (a pilot-scale current Good Manufacturing Practice, or cGMP, facility under construction at the University of Delaware, groundbreaking April 2024) exist precisely to de-risk this kind of modern, data-rich manufacturing before it reaches commercial lines. A reproducible, profile-gated dev stack is how you experiment with that architecture without a cleanroom — and the same Compose-and-Make discipline scales straight into the IQ/OQ evidence a real facility would demand. The honest gap is in the operational qualities — uptime guarantees, certified support, formal validation packages — not in the shape of the data plant.

Key terms

• Container / OCI image — a sealed, portable bundle of an application and its dependencies, identified by an immutable content digest; the unit each service ships as.
• Docker Compose / compose.yaml — the declarative file (and the tool) that defines a multi-service application and brings it all up with one command.
• Profile — a Compose label that gates which services start, so a reader only powers on the layer the current chapter needs (core, capture, semantics, trust, analytics).
• Tag pinning — fixing an image to a specific MAJOR.MINOR.PATCH version (and digest) rather than :latest, so the environment is reproducible and upgrades are deliberate.
• Healthcheck — a command the platform runs to decide whether a service is actually ready, so dependents (and tests) can wait correctly.
• Historian — the time-series store for high-rate process data; here, a TimescaleDB hypertable inside the same PostgreSQL database as the batch model.
• Hypertable — TimescaleDB's PostgreSQL table that is automatically partitioned by time into chunks for fast time-series writes and queries.
• SIM_SEED=2026 — the fixed master seed that makes the CHO simulator's output byte-for-byte identical everywhere, so datasets are reproducible and CI can verify them.

Where this leads

The stack is up and the simulator's numbers are sitting in PostgreSQL — but right now a row like BR101.DO.PV = 48.6 is just a float with a label. To make it mean something, we need the skeleton that says which equipment, which recipe, which phase, and which batch that reading belongs to. The next chapter, The Batch & Equipment Data Model: ISA-88/95 in PostgreSQL, builds exactly that relational backbone — the model that turns every later number into a fact about a batch.