Separating Tools from the Orchestrator — A Design Principle for Personal Software Ecosystems

The accumulation of personal software grows over years. A standalone script first, then a more elaborate CLI, then a polished tool with a GUI, then a workflow that strings several together ── and at some point this collection grows large enough to be reasonably called a personal software ecosystem.

A design judgement that gets overlooked through that growth is the recognition that the elements of the ecosystem belong to two distinct layers. One is the tool layer that solves an individual problem; the other is the orchestrator layer that composes tools into an end-to-end workflow. Treating them as deliberately separated layers is the design judgement that supports the ecosystem’s long-term durability and composability.

This article articulates that design principle, walks through implementation patterns at the orchestrator layer, and uses the writer’s own open-source ecosystem² as a concrete instance.

The Two Layers Defined

The two layers separate cleanly by responsibility and interface.

Tool layer:
– Single responsibility ── a clear single purpose: “set chapter markers on a video”, “rectify a slide photograph’s trapezoidal distortion”, “typeset a LuaTeX document”
– Owns a data format ── canonical for the format it produces (.vce.json, corrected PNG, PDF)
– Independent release ── manages its own version, release cadence, and dependencies
– Independent distribution ── pip-installable, standalone binary, distribution channel not coupled to other tools

Orchestrator layer:
– Composition is the responsibility ── stitches existing tools together via data flow to achieve a target outcome
– Calls tools as dependencies ── pip imports, subprocess calls, CLI invocation; never absorbs tool implementations into its own codebase
– Owns the workflow logic ── stage ordering, conditional branching, error handling, retries, logging
– Stays thin ── thickening is a sign that tool-layer logic is being duplicated, and re-separation is overdue

The boundary between the two layers is drawn by standardised data formats. SRT, PNG, PDF, JSON, YAML ── as long as long-stable, open formats serve as the inter-layer interface, tool-layer components can be replaced without touching the orchestrator, and the orchestrator can call tools regardless of their implementation language.

Why Separate?

Four structural benefits emerge from this separation.

First, replaceability. With tools as independent components, swapping one out for another comes at minimal cost. If a transcription stage moves from Whisper to a different model, the orchestrator sees only the SRT format at the boundary; the tool-layer swap completes the change.

Second, parallel evolution. Tools as independent repositories evolve on independent release cycles. GUI improvements, format version bumps, performance work ── each proceeds without disturbing the others. The structural slack this creates is what allows year-on-year accumulation without it congealing into a single brittle codebase.

Third, external reuse. Each tool retains value outside the workflow it was originally built for. perspective-corrector is part of a meeting-record pipeline, but the same tool also has independent value as an app for fixing whiteboard photos taken with a phone. Strict single-responsibility unlocks unexpected reuse contexts.

Fourth, distributed maintenance burden. The orchestrator stays thin; the tools stay tightly scoped. Maintenance load on either layer is lighter, both cognitively and code-wise, than carrying a layer-mixed monolith forward over years.

The Interface Honours Existing Standards

The choice of data format at the layer boundary follows the principle articulated in the dlab essay Standardising Records ── individual tools should not invent standards; they should respect existing standards and connect through them.

Data nature	Standard format	Reason
Time-coded subtitles / transcripts	SRT	Default since the 1990s; every modern speech-recognition tool emits it
Images (rectified slides etc.)	PNG / JPEG	Published spec, multiple implementations, long-term readability
Structured metadata	JSON / YAML	Language-neutral, human-readable, git-friendly
Typesettable source	LaTeX / Markdown	Open formats, multiple implementations, 30-year-class durability
Distribution artefact	PDF/A	ISO 19005, designed for long-term preservation
Video chapter notation	YouTube chapter syntax / .vce.json	Widely accepted / project-specific

A new format is invented only when the existing standards genuinely cannot represent the information model in question. .vce.json is one of the few cases ── chapters + source-file boundaries + encoder configuration form an information model that doesn’t fit any existing standard, so video-chapter-editor defines its own and owns it.

Tool Selection ── Single Responsibility, Strictly

The design judgement at the tool layer reduces to the proposition Doug McIlroy stated in the Bell System Technical Journal in 1978³:

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”.

The operational form of the judgement is simple: when a new feature is being considered for an existing tool, ask “what would be lost if this were a new, independent tool instead?” If nothing significant is lost, separate. This is McIlroy’s proposition translated into a day-to-day decision rule.

Examined under this rule, the writer’s currently maintained tools sort cleanly:

Tool	Single responsibility	Format owned
video-chapter-editor	Set / edit chapters on a video	.vce.json (chapters + boundaries + encoding settings)
perspective-corrector	Correct trapezoidal distortion in slide photographs	Rectified PNG / A4 PDF
luatex-docker-remote	Run LuaLaTeX typesetting via remote Docker	Input LaTeX → output PDF

Each tool’s responsibility can be stated in a single line. When that’s no longer true, the tool is a candidate for splitting.

Implementing the Orchestrator Layer

Choosing the language for the orchestrator layer often presents itself as a single-answer question: Python, or zsh, or YAML, or Make? The framing is misleading. Each is the most mature tool for a different layer of the orchestration stack, and the productive answer is to layer them rather than to pick one.

Tool	Strength	Role
Make	POSIX-stable since 19794. Declarative file dependencies + idempotent execution	Pipeline skeleton (DAG)
Python	Tests, debugger, exceptions, type hints. Complex transformation / branching	What Make can’t express cleanly (template generation, metadata transforms)
YAML	Declarative, language-neutral, git-friendly	Human-edited configuration (encoder presets, profiles)
zsh / POSIX shell	Most natural CLI composition syntax	Shell wrappers, aliases, light runners

External workflow platforms (n8n, Argo Workflows, etc.) earn their keep on cross-service integration + event-driven automation. They are overkill for a linear CLI pipeline, and the platform dependency they introduce works against the 30-year durability principle articulated in Designing Time-Resilient Assets.

A typical Makefile (skeleton in five lines):

report.pdf: report.tex
    luatex-pdf $<

report.tex: subtitles.srt slides_corrected/
    msw-report --srt subtitles.srt --slides slides_corrected/ -o $@

subtitles.srt: $(RECORDING)
    yt-srt $< -o $@

slides_corrected/: $(SLIDES_DIR)
    perspective-corrector $< -o $@

A single make report.pdf runs only what’s missing or out-of-date and skips the rest. The essence of the orchestrator layer ── stitching existing good tools together through file dependencies ── lands directly on Make’s 47-year-stable semantics⁴. Where logic, branching, or API calls are needed, those move to Python; configuration that humans edit moves to YAML. Layering this way lets the orchestrator itself satisfy a don’t-write discipline.

From the Current State to the Target Structure

media-scribe-workflow currently mixes tool and orchestrator code in a single repository: the GUI (media_scribe_workflow/ui/), the encoding CLIs (vce-*), various utilities (yt-srt, video-trim, video-chapters), and the report pipeline (msw-*) all live in the same package.

The target structure is:

Independent tool repos:
  - video-chapter-editor      (GUI + .vce.json format owner + vce-encode/split)
  - perspective-corrector     (image correction, already separate)
  - luatex-docker-remote      (typesetting environment, already separate)
  - (future) subtitle-utils   (yt-srt and similar small tools)

Orchestrator repo:
  - media-scribe-workflow     (depends on the above, thin orchestrator only)
    ├─ Makefile               (DAG skeleton)
    ├─ msw_pipeline/          (Python helpers ── template generation, metadata transforms)
    └─ profiles/*.yaml        (human-edited configuration)

The migration path is to use git subtree split on video-chapter-editor to preserve commit history into a new repo, then promote shared modules (pipeline.srt_parser, a few helpers in utils) either into a common library or duplicated into each consumer repo. None of this needs to happen at once; splitting video-chapter-editor first and accumulating operational evidence before extracting common modules is the staged approach.

---

Read against the broader dlab corpus, the tool / orchestrator separation is the structural integration point for several arguments developed in earlier essays.

It is, above all, the most structural implementation of the stance declared in the foundational essay Relative, Not Deterministic ── do not treat a tool as “the correct procedure”; make the range it mediates explicit, and let the tool itself be swapped relatively as the situation changes¹. Closing a tool to a single responsibility is narrowing what it mediates so that the “shape of thought” it prescribes is predictable and substitutable; fixing the boundary (SRT) while keeping the tool relative is “relative, not deterministic” in its most concrete form.

The reuse principle from The Discipline of Reuse — Designing by Not Writing operates as the design rationale for the orchestrator layer’s don’t write discipline. Whisper, Claude, LuaTeX, ffmpeg, Qt are all good programmers’ code maintained by others; the orchestrator writes only the connective tissue ── which is also a choice of mediation: entrusting the mediation you don’t write to other people’s excellent implementations.

The inter-stage interface principle from Standardising Records is precisely the boundary between the tool and orchestrator layers. Honouring established formats (SRT, PDF/A, JSON) is what guarantees substitutability on both sides, and makes Verification ── mechanical difference detection between adjacent stages ── easy to run.

The 30-year durability argued in Designing Time-Resilient Assets is realised when the tool layer is single-responsibility-tight and the orchestrator runs on long-stable substrates (Make, POSIX, Python).

The porcelain / plumbing decomposition Git made famous⁵ anticipates the present article’s claim, applied at the level of command-line tools. Porcelain is the orchestrator (the surface polished for human use); plumbing is the tool layer (the mechanical connection points); the two are bound by stable interfaces.

Designing an ecosystem turns out not to be about writing code so much as drawing the boundaries of mediation. Where to cut for single responsibility, which formats to elevate to inter-layer interfaces, where to apply the don’t write discipline at full strength ── these judgements are the operational core of converting accumulated personal software into a structurally maintainable ecosystem. In a teaching context, the decision-making process behind these boundary judgements is what holds value far longer than the operation of any specific tool.

References