ADR 0004 — Lifecycle states in the diagram index

Status: Accepted, implemented

Implementation summary:

• status: is parsed into a LifecycleState with five values: draft, in-review, publish, deprecated, archived. An unrecognised value fails the run.
• All five states are converted by default, each emitting its own adms:status in the provenance graph — see Amendment 2, which supersedes the two-group selection rule this ADR originally decided.
• --exclude-states names states to leave out, and cannot name all of them. --include-states and --include-drafts are accepted, ignored and warned about.
• Concept IRIs come from the ADMS status vocabulary; no term is minted in the arch: namespace.
• supersededBy: on a retired entry names its replacement, emitted as dct:isReplacedBy / dct:replaces and deliberately without owl:sameAs.
• The states form a validated lifecycle. identity-lock.yaml records the state each entry was last seen in, an illegal transition fails the run, and --allow-state-change authorises one.
• --no-render-archived stops drawing archived entries without affecting the graph.

Scope: what status: may say, which values are published, and how a state reaches the graph.

Depends on: ADR 0001, which assigns status: to the publisher as the selection concern, distinct from identity and from descriptive metadata.

Related: ADR 0003 governs a change of identity. A change of state is a different event: the IRI stays put and what is said about it changes. The amendment below extends that ADR's lock file to record state as well, for the same stated reason.

ADR 0011 governs architectureState: — which reality a diagram describes, baseline or target. The two are the most confusable pair in the index and answer different questions: a target-state diagram can be perfectly published. That ADR sets out the distinction, and why it inherits and defaults differently from this one.

Context

status: was a free string. One literal had an effect:

return if (includeDrafts) entries else entries.filter { it.status != "draft" }

Three consequences followed, none of them intended.

Any other value published. status: archived published an archived diagram as current. status: Draft published a draft, because the comparison was case-sensitive. status: publsih published, so a typo in draft was indistinguishable from publish — the failure mode was silent and in the wrong direction.

The lifecycle never reached the graph. status was the only ModelMetadata field emitProvenance did not emit. A consumer querying the triplestore could not tell a current diagram from one kept only for the record. LinkedArchiEmitter constructed ModelMetadata(status = "") for non-indexed runs, which was harmless precisely because nothing read the field.

There was no way to express the states people actually use. A diagram awaiting approval, one superseded but still linked, one kept for the record — all three had to be spelled either draft or publish, and neither fits.

Decision drivers

The states divide on one question, and it is not "how finished is this?" but has an IRI for this diagram ever been public?

An unreleased diagram must not be published: converting a draft mints IRIs and writes an SVG for something whose author has not finished deciding what it is. Excluding it is the whole point of having a status.

A released diagram must not be un-published. Its IRIs are already linked from documentation, from other models, and from downstream graphs. Removing it from the output does not communicate "deprecated" — it produces a dead link, which communicates nothing and is precisely the harm the formerIds machinery in ADR 0003 exists to avoid. Deprecation is a statement about a resource, so the resource has to be there to carry it.

That gives two groups with different treatment, which a boolean cannot express and an enum can.

Options considered

A. Keep the free string, add an allow-list only

Validate the value, keep the single "is it draft" exclusion rule.

Catches the typo, but leaves deprecated and archived meaning exactly the same thing as publish — accepted, published, unrecorded. The state is checked and then discarded, so a consumer still cannot tell current from superseded.

B. Exclude everything that is not publish

The tidy reading: only published things are published.

Wrong for the post-publication states, and quietly destructive. status: deprecated would delete the diagram's IRIs from the graph on the next run, breaking every inbound link at the moment the author was trying to signal "still here, don't build on it". It also makes archived useless: an archive nobody can address is a deletion.

C. Two groups, and the state published as RDF

Pre-publication states are excluded. Post-publication states are included and annotated.

Costs a vocabulary decision and one more concept for authors to hold. Buys a lifecycle that is visible to consumers, and makes retiring a diagram non-destructive.

Decision

Adopt C, with five states.

status:	Selected by default	adms:status	Why
draft	no	UnderDevelopment	never published; nothing links to it
in-review	no	UnderDevelopment	submitted, not approved; same
publish	yes	Completed	the default when status: is absent
deprecated	yes	Deprecated	published; links exist and must resolve
archived	yes	Withdrawn	published; kept for the record

Superseded. The "Selected by default" column no longer describes the implementation: every state is converted. See Amendment 2. The adms:status column stands.

1. An unrecognised value fails the run, naming the value, the file, the index and the accepted set. Failing closed is the point: the behaviour being replaced was to publish.
2. Case and separators are normalised, and published and obsolete are accepted as synonyms. Both would have published silently before, so recognising them is strictly safer than refusing them, and rejecting a run over a synonym buys nothing.
3. The whole index is validated before entries are filtered out, so a typo on a draft entry fails on the run that introduced it. This matches the existing rule for ids.
4. ~~--include-states adds to the default set rather than replacing it. A preview build wants the drafts and the live diagrams; a build that shows only drafts has no use anyone asked for. Narrowing the published set is done by editing the index, which is where a publication decision belongs.~~ Superseded by Amendment 2: every state is converted, and --exclude-states withholds one.
5. ~~--include-drafts stays, as a deprecated alias for --include-states draft, so existing pipelines keep working.~~ Superseded by Amendment 2: accepted, ignored and warned about.
6. A declared state is emitted; an absent one is not. status: omitted means the entry is published and the graph says nothing about its maturity. Emitting Completed because nobody said otherwise would put a claim in the graph that no author made.
7. The status describes the view, so it sits beside that entry's title and dates. A status: on the model: entry is a default for its views and is emitted on each of them, so one line can retire a model without the model itself being the subject of a claim about diagram maturity.

Vocabulary

adms:status from ADMS, with concepts from http://purl.org/adms/status/:

<https://example.org/la/plantuml/order-domain/graph/provenance> {
    <…/view/current> adms:status <http://purl.org/adms/status/Completed> .
    <…/view/leaving> adms:status <http://purl.org/adms/status/Deprecated> .
}

Chosen over the alternatives considered:

• A new arch:lifecycleState. Rejected: it would need an ontology release and a SHACL shape before it could be validated, and it would make the lifecycle legible only to consumers that already know this project. adms:status is read by anything that reads DCAT-AP.
• dct:accrualStatus. Rejected on semantics. Despite the promising name it describes the accrual policy of a collection, not the maturity of a resource. DCMI has no general-purpose status property, which is why ADMS defines one. (Correcting a suggestion made earlier in the discussion that led to this ADR.)
• owl:deprecated. Rejected: it annotates ontology terms, and an arch:View is data, not vocabulary.
• A local SKOS scheme mapped to ADMS with skos:exactMatch. Rejected as unnecessary. It exists to preserve precision ADMS cannot express, and the only such case is draft versus in-review, which both map to UnderDevelopment. Neither is selected by default, so the only build in which either appears asked for it by name and already knows which. The three states that reach a published graph map one-to-one.

dct: remains the vocabulary for the things it does define, and already carries the neighbouring facts: dct:title, dct:created, dct:modified, dct:description, and dct:isReplacedBy / dct:replaces for the identity aliases of ADR 0003.

Consequences

• A consumer can distinguish current, deprecated and archived diagrams with one triple pattern, and can do so using a vocabulary it may already support.
• Retiring a diagram is non-destructive. Published IRIs keep resolving, and render keeps writing the SVG, so documentation links do not rot.
• A status typo fails the run instead of publishing. This is a breaking change for an index carrying a value outside the accepted set: such an index published before and now fails. That is the bug being fixed, and the message names the file and the accepted values.
• Authors have five states to choose between instead of two. The table above is the whole contract.
• --include-drafts is deprecated but functional, so no pipeline needs changing at once.
• The adms: prefix appears in every output, registered alongside dct: and skos:.

Amendment — the three questions this ADR left open

All three are now decided and implemented. They are recorded here rather than in a separate ADR because each one only makes sense as part of the state model above.

1. A retired diagram names its successor

Decided: yes, with supersededBy:.

A status says a diagram is on its way out. Without a successor a consumer learns that something is retired but not where to go, which is the more useful half of a deprecation.

views:
  - id: order-flow-v1
    file: order-flow-v1.puml
    status: deprecated
    supersededBy: order-flow-v2

<…/view/order-flow-v1> dct:isReplacedBy <…/view/order-flow-v2> .
<…/view/order-flow-v2> dct:replaces     <…/view/order-flow-v1> .

No owl:sameAs, and that is the whole reason this could not simply reuse the formerIds: machinery from ADR 0003. A former id is the same resource under a previously published name, so asserting identity is correct. A successor is a different diagram; asserting owl:sameAs would merge the two, and a reasoner would then merge their elements as well. Same predicate family, opposite claim about identity, so Succession is separate from IdentityAliases.

supersededBy: is read at the level the entry sits on, like formerIds:. Four things are refused, all so that following dct:isReplacedBy arrives somewhere real: a successor that is not in the index, one in a state a default build withholds, an entry naming itself, and a cycle. A fifth rule requires the state to be deprecated or archived, since "current, and also replaced" is not actionable and usually means the two entries were swapped.

Succession across models is not expressible, because the reference is resolved inside the index. Deliberate: an unresolvable successor is worse than none, and cross-repository references are the aggregation repository's concern (ADR 0001, decision 2).

2. Archived entries can stop being rendered, by choice

Decided: an opt-in flag, --no-render-archived.

Not a default and not a timeout. The default keeps drawing archived diagrams, because the reason they stay in the graph — published URLs that should keep resolving — applies to their images too. But an output directory meant to show only what is in use is a legitimate thing to want, so the publisher can ask for it.

The flag affects render only. convert still emits the triples, so the diagram keeps its IRIs and nothing that links to it in the graph breaks; only the image stops being written. That split is the point: the graph is the record, the SVG directory is a presentation of it.

This deliberately does not answer the equivalent question ADR 0003 leaves open for formerIds aliases. A superseded name and a retired diagram are different things, and a time-based rule for either is still unattractive: it would make output depend on the clock.

3. draft and in-review differ, and the lifecycle enforces it

Decided: distinct meanings, and a validated state machine.

Both states are withheld, so the difference cannot be about what gets published. It is about whose turn it is: draft is work in progress with nobody waiting on it, in-review is submitted and waiting for validation.

That makes in-review the gate into publication, and — the case that gives the distinction teeth — the only state a published diagram can return to without being retired:

From	May become
draft	in-review
in-review	draft, publish
publish	in-review, deprecated, archived
deprecated	publish, archived
archived	publish, deprecated

Two moves are refused. Anything published back to draft, because its IRIs are already public and "unfinished" is not a thing a public resource can be. draft straight to publish, because that skips the validation in-review exists to require. Nothing is a dead end, so a retired diagram can always be reinstated.

Enforcing a transition needs the previous state, and an index states only where a diagram is now. This is precisely the argument ADR 0003 makes for the identity lock — "detecting a change requires a record of what was published" — so the same record answers it. identity-lock.yaml gains a status: per entry. A state the record has never seen is not a move, so adopting states needs no migration: the first run records, the next enforces. --allow-state-change authorises a refused move, mirroring --allow-identity-change.

Withheld entries are reconciled too, which is the part that is easy to get wrong. The transitions that take a diagram out of the output are exactly the ones whose entry is then not converted, so checking only converted entries would skip the moves most worth checking, and publish → draft would silently drop a published diagram. Withheld entries therefore have their transition checked, and their new state recorded — while keeping the recorded ids, since nothing new was published and overwriting them would let a rename of a published IRI slip past the identity check on the run that eventually republishes it.

Sending a live diagram back to in-review is legal and breaks links, so it warns rather than fails:

[WARN] 'order-flow.puml' was published as 'publish' and is now 'in-review', so it is left out of
this run and its IRIs stop resolving. Add --include-states in-review to keep publishing it while
it is out of use.

Recording the new state is also what makes that warning fire once, on the run where the link actually breaks, rather than on every run thereafter.

Consequences of the amendment

• A consumer can follow a retirement to its replacement in one hop, using a predicate it already knows, without being told two different diagrams are the same resource.
• The lifecycle is enforced rather than advisory, so in-review is a gate instead of a label.
• identity-lock.yaml now carries state as well as identity. A pipeline that discards it loses transition checking as well as rename detection, and reverts to accepting any state.
• Backstage has no identity lock, so it gets the states and supersededBy: but no transition checking. Worth closing, and out of scope here.
• Two more escape hatches exist (--allow-state-change, --no-render-archived). Both are opt-in and both are recorded in the message or the docs as having a cost.

Amendment 2 — every state is converted by default

Status: Accepted, implemented. Supersedes decision points 4 and 5 above, and the "Selected by default" column of the decision table.

The original decision put the states into two groups: pre-publication states were withheld, post-publication states were kept and annotated. Selection and description were answered by the same property. That is now one question, answered once: status: describes a diagram, it does not decide whether the diagram exists. All five states are converted, each carrying its own adms:status, and a consumer that wants only current diagrams filters on that in the triplestore.

Why the original split did not hold

The two-group rule was justified by "an unreleased diagram must not be published". Three things undermined it in practice.

It withheld the diagrams most in need of review. A draft is exactly the thing a reviewer wants to see in a graph — cross-referenced against the models it touches, checked by the same SHACL shapes as everything else. Withholding it means the first time a diagram meets validation is the run that publishes it, which is the wrong moment to discover a problem.

Selection was the wrong instrument for the concern. The output is a graph, and a graph carries its own metadata. adms:status already says a diagram is unfinished, in a vocabulary DCAT-AP consumers understand. Withholding the triples as well as marking them says the same thing twice, and the second copy is the one that cannot be queried.

The failure direction was wrong. An inclusion list omits by default, so the mistake it invites is a state nobody named — diagrams silently missing, discovered later as dead links in documentation. An exclusion list publishes by default, so its mistake is a state nobody excluded — something published early, which is visible immediately and fixed by a rerun. Between a silent omission and a visible over-share, the graph should fail towards over-sharing, because that is the one an author notices.

What changed

1. DEFAULT_SELECTION is every state. A run converts what the index lists.
2. --exclude-states replaces --include-states as the only thing that withholds an entry. It refuses to name all five states: a run that converts nothing writes an empty graph and exits zero, which is indistinguishable from success.
3. --include-states and --include-drafts are accepted, ignored, and warned about. They only ever added to the processed set, and that set is now everything, so anything they could have named is already in — which makes ignoring them safe. Reinterpreting --include-states as "only these" was rejected: it would silently narrow the output of every pipeline that passes it, which is the failure direction this amendment exists to avoid.
4. selectedByDefault became published. The old property conflated "is this converted?" with "has an IRI for this ever been public?". The second question survives on its own, because two things still depend on it and neither is about selection: the transition table refuses to send a published diagram back to draft, and supersededBy: refuses to point at something unreleased. Had it stayed one property, flipping the default would have made both checks vacuous.

Consequences

• A draft's IRIs are minted. That is the substantive cost, and it is real: an IRI exists before its author has finished deciding what it means. It is accepted because adms:status says so plainly, and because the alternative hid drafts from the validation that would catch their problems. A publisher who cannot accept it excludes the state.
• supersededBy: keeps its rule for a new reason. Pointing at a draft was refused because a default build minted nothing for it and the reference would dangle. It is still refused, but now because the advice is wrong: "read this instead" should not send a reader to unfinished work.
• The withheld-entry warning survives, narrowed. It fires only in a build that passes --exclude-states, which is the only build where a diagram can leave the output without anyone editing it. Without the flag there is no link to break, so there is nothing to say.
• --no-render-archived is unaffected, and can now empty the render set when combined with --exclude-states; that combination is refused for the same reason as excluding all five.
• Existing pipelines keep working. A pipeline passing --include-states draft,in-review gets the same diagrams it got before, plus a warning that the flag is no longer needed. A pipeline relying on drafts being withheld gets more than it did, and must add --exclude-states. That is a breaking change in behaviour without one in syntax, which is why it is called out in the changelog.

Open questions

• Should the lifecycle be configurable? The transition table is compiled in. A repository with a heavier review process might want more states or stricter moves, which would mean a schema for them rather than an enum.
• Should --exclude-states be settable in the index? A repository that always withholds drafts must pass the flag on every invocation, and a forgotten flag publishes. A default in the index would make the decision travel with the models, at the cost of a second place to look.
• Should render warn when it withholds an archived image that was published before? convert warns in the equivalent case, using the lock. render does not read the lock at all, so --no-render-archived currently breaks image URLs silently.
• Should Backstage get an identity lock? It is the one index-aware converter without one, so it validates neither renames nor transitions.