Fountain-Store.git · PLANS.md
Fountain-Store.git / PLANS.md
revision 7f73dea8c1cf9fc89612d1d7172f86b86cb11596 · complete file
# PLANS
## Self-hosted SemVer repository authority and recoverability — 2026-09-02
**Capability.** Move the release-managed Fountain Coach source repositories and their recoverable version history
from GitHub to the owner-controlled Hetzner estate, with a secure, independently verifiable backup and restore path.
**Acceptance proof.** The selected repository set has authoritative bare repositories on the owner-controlled host;
each selected local checkout can push and clone over locked SSH without GitHub; every repository and required release
ref has a matching signed/digested recovery manifest; an encrypted backup is written to independent custody; a fresh
restore reproduces refs and manifests; and only then are GitHub remotes and GitHub-only dependency URLs removed from
the selected operational path.
**Chapters read — phase A.** 07 (plan before high-risk edits and evidence-backed claims); 08 (separate repository,
Store, and recovery evidence); 116 (FountainStore remains publication authority and edge); 117 (typed export,
production recovery projection, independent mirror, and non-authoritative Git); 118 (continuity, confidentiality,
and recoverability). The relevant FountainStore AGENTS.md also binds pure Swift, stable APIs, and deterministic
validation.
**What they forbid here.** Treating a Git working copy as the authority; using the Git repository as a second
FountainStore or publication authority; putting credentials into repositories or manifests; declaring a backup from
a file copy alone; or deleting GitHub references before clone/restore and dependency-provenance proof.
**Conflicts.** Chapter 117 Rule 3 and its governing sentence currently require a private GitHub off-host mirror;
the writer now requires zero GitHub dependency. This must be resolved by an explicit governance amendment before the
replacement mirror can be accepted. The Hetzner volume is attached and protected, but same-host storage is not yet
independent custody.
**Excluded, and why.** Repository creation, remote replacement, dependency mirroring, GitHub removal, and backup
automation remain deferred until the exact independent backup custody is selected and the Chapter 117 conflict is
amended. The 50 GB Hetzner volume `106773053` is provisioned at `/mnt/HC_Volume_106773053`; it is staging storage,
not yet recovery proof.
**Current evidence.** Hetzner CLI context `fountainai` sees `ubuntu-2gb-hel1-1` at `65.109.14.71`; the volume is
attached, ext4, and delete-protected. The host has 1.7 GB free on its root disk, no Git service, no `git` account,
and no backup timer. The migration is **BLOCKED — independent backup custody and governance replacement are not yet
established**.
## FCIS-KIT remote credential handoff receiver — 2026-08-31
**Capability.** Complete the operational receiver for the released
`fountainstore.credential.provision.remote` FCIS-KIT instrument.
**Proof.** The Swift HTTP server accepts only the exact `/agent/v1/provision-credential` route; authenticates the
enrolled host credential through `FountainStoreHostAgentService`; rejects wrong target, identity, expiry, replay,
admission, remote SecretStore reference, and fingerprint; writes the Store credential only to the configured
server-side SecretStore account; returns the FCIS-KIT redacted terminal receipt; and focused tests plus the server
build pass. Production activation remains separate until the signed server artifact is deployed and remote
read-back proves the receiver.
**Chapters read** — 07 bounded implementation and STOP discipline; 08 evidence ownership; 91 FCIS-KIT boundary;
94 credential custody and owner-controlled operational production; 97 enrolled host-agent boundary; 122 semantic
transformation claim boundary.
**What they forbid here** — API keys in typed requests, MIDI2, receipts, logs, or command arguments; bypassing the
enrolled host agent; arbitrary SecretStore account writes; global API-key fallback for the bootstrap route; and
claiming production handoff from a local fixture.
**Excluded, and why** — provider API provisioning, SSH deployment, DNS/Caddy changes, and Chapter 122 publication;
they require their own exact target and evidence gates after this receiver is released.
## v0.4.1 patch release — 2026-08-28
**Release scope:** publish the estate-publication reload fix on the canonical `v0.4.1` SemVer tag. The release
metadata is aligned with the existing `0.4.0` mainline release; the older `0.3.x` README and version strings were
stale.
**Acceptance:** release builds, focused estate regression, clean diff, pushed `main`, and pushed `v0.4.1` tag. The
full suite's two existing HTTP backup-pagination tests currently fail because their embedded test server becomes
unreachable mid-test; the failures reproduce without this release slice and are not treated as estate regressions.
No deployment or production mutation is included.
## Estate publication snapshot reopenability — 2026-08-28
**Objective:** make the native HTTP server load the durable estate publication snapshot after a fresh
`FountainStore` process opens the Store.
**Root cause:** `EstatePublicationStore.loadCurrent` used `listCollections()`, which reports only typed collection
hooks registered in the current process. A fresh server therefore rejected a valid persisted snapshot as missing.
**Scope:** remove the process-local preflight, keep the typed read read-only, and add regression coverage for both an
absent snapshot and a snapshot loaded after Store reopen. No schema, storage format, or server fallback changes.
**Validation:** focused `FountainStoreHTTPTests.EstatePublicationTests`, HTTP server build, and live readback against
the prepared Reframe estate Store.
## Book Library admission into the single FountainStore authority — 2026-08-26
**Chapters read** — 07 (bounded slice, focused validation, and evidence before claims); 08 (process, Store, and
release evidence are separate authorities); 55 (Store choice and persistence are explicit Swift authority);
56 (Book Library remains the separate portable provider and FountainStore is persistence after import); 95 (the
headless server exposes the existing Store authority and does not create a second storage engine).
**What they forbid here** — copying provider curation or publication authority into FountainStore; starting a second
provider service to satisfy Store reads; guessing a work, Store, endpoint, or release; or treating a successful fetch
as publication approval.
**Conflicts** — none. The writer's request for one deployed FountainStore authority is implemented as explicit
provider fetch followed by durable Store admission, preserving Chapter 56's separate provider repository boundary.
**Excluded, and why** — public edge cutover, TLS/Caddy changes, provider promotion, credentials, and remote Store
writes are excluded from this package slice; they require separate deployment and acceptance evidence.
Objective: consume the pinned `Fountain-Coach/book-library` SemVer release through its reusable client library and
persist one explicitly selected published work in FountainStore as one durable record.
Validation: client and importer focused tests, package resolution at `book-library` 0.4.0, and diff checks.
## Governance amendment — owner-controlled operational production (2026-08-23)
The paired MIDI2 governance amendment introduces operational production as a distinct owner-controlled tier. It may
proceed only after a named provider adapter, SecretStore custody, exact target, pinned/signed artifact, TLS/host
handoff where applicable, rollback/revocation evidence, and bounded live acceptance. Independent security review is a
later assurance gate for security-reviewed/public claims and is not represented as complete here.
## Production-readiness evidence package — bounded internal slice (2026-08-23)
**Objective:** assemble a reviewable production-readiness dossier for the accepted `59a6219` candidate and identify
the independent artifacts still required before any production mutation.
**Scope:** local full-suite/build evidence, pinned dependency/provenance summary, production-shaped unit and rollback
review, staging acceptance references, and an explicit external-sign-off checklist. No production service, DNS, Caddy,
TLS, provider, credential, or security-review mutation is included.
**Acceptance:** the dossier must distinguish established staging evidence from unestablished production authority and
must end in GO/NO-GO with named missing artifacts; it cannot manufacture provider authorization or an independent
security review.
**Result:** `docs/PRODUCTION_READINESS.md` records a truthful NO-GO for source `59a6219`. The full local suite passed
114 tests with 0 failures; release builds for `FountainStoreHTTPServer` and `FountainStoreDoctor` passed; candidate
and lockfile digests, staging evidence, rollback boundary, and four missing external gates are named. No production
mutation was performed.
## Crash-safe staging lease recovery — bounded implementation (2026-08-23)
**Objective:** allow an explicitly opted-in supervised staging server to recover one provably stale owner lease
before selecting its exact Store, so systemd restart-on-failure can reopen after an unclean process death.
**Scope:** add a SessionKit recovery policy that reuses the existing compare-and-remove stale-owner authority; wire the
HTTP server to it only when `FS_RECOVER_STALE_LEASE=1`; add deterministic tests and document the staging-only setting.
**Constraints:** live owners, malformed owners, changed owners, and disabled policy must still fail closed; no path
searching, lease guessing, credential handling, production-unit change, or automatic repair by the doctor tool.
**Validation:** SessionKit tests cover stale recovery and live-owner refusal; HTTP server tests/build and diff checks
pass; the remote staging candidate is rolled only after a pinned release artifact is built and locally verified.
**Local result:** `FountainStoreLeaseRecoveryPolicy` defaults to `.refuse`; the HTTP server opts into `.recoverStale`
only when `FS_RECOVER_STALE_LEASE=1`. Ten SessionKit tests pass, including stale recovery, default refusal, and live
owner refusal; the focused HTTP suite (24 tests), server build, and `git diff --check` pass. Remote rollout remains
pending a pinned Linux artifact.
**Staging result:** candidate `59a6219` was built on Ubuntu 24.04 with Swift 6.3.3 and deployed as
`/srv/fountain-store/releases/0.4.0-staging-59a6219/fountain-store-server`, SHA-256
`2815aa7e4e4a4563852b7f3fca9f09740863e2df45dee99d180c21b48d3d55ce`. The staging unit enabled the explicit recovery
policy and remained disabled for boot; normal readiness, record readback, and doctor integrity passed. A bounded
`SIGKILL` then produced a new PID, systemd restart count 1, ready HTTP state, preserved record, live owner, and doctor
integrity pass without manual repair. Caddy/DNS/TLS/provider/production/security-review boundaries remain unchanged.
## Fresh candidate persistence acceptance — bounded mutation (2026-08-23)
**Scope:** on candidate `59a6219`, use one new record in the existing isolated acceptance collection, create one
backup, mutate the record, restore the backup, restart the supervised unit, and verify readiness, record recovery,
identity, and doctor integrity. No other collection or service is targeted.
**Exclusions:** production activation, public routing, TLS/DNS/Caddy, provider mutation, credential changes, and
independent security review.
**Result:** candidate `59a6219` created/read record `codex-staging-59a6219-1787480723-26207`, created backup
`7D4A6234-3C13-47A0-ADB0-FE0C7E791DA0`, observed the post-backup mutation, accepted restore with HTTP 202, and after
supervised restart returned the backed-up `before-restore` value. Readiness remained 200 with stable server identity;
the release doctor reported source revision `59a6219`, live owner, three backups, and integrity pass. The immediate
in-process read after restore still showed the post-backup value, so durable restore/reopen—not instant cache
invalidation—is the accepted boundary.
## Chapter 95 implementation slice — typed headless server authority (2026-08-23)
**Objective:** promote the existing HTTP server with a typed, testable configuration and explicit live/ready
authority boundary while preserving the FountainStore engine and existing routes.
**Scope:** configuration precedence for `FS_PATH`, `FS_BIND_HOST`, `PORT`, and authentication mode; loopback-safe
default binding; persistent server identity; `/health/live` and `/health/ready`; readiness state transitions; and
focused tests plus contract documentation.
**Constraints:** the Store engine remains the persistence authority; the server must not create a second Store; no
credentials enter responses or logs; existing `/health` remains backward-compatible; no public-production or Linux
interoperability claim is made from macOS tests.
**Phases and acceptance:**
1. Add pure Foundation configuration and identity/readiness types. Acceptance: deterministic unit tests cover
defaults, environment overrides, invalid values, canonical paths, and identity reuse.
2. Wire the server to the typed configuration, session/lease authority, and readiness endpoints. Acceptance: the
server opens one canonical Store, marks readiness only after open and authority admission, and releases its lease
on bounded shutdown.
3. Update the HTTP contract and docs. Acceptance: the OpenAPI document distinguishes liveness from readiness and
documents the loopback-safe default.
4. Run focused HTTP/session tests, package build, and diff/leak checks. Acceptance: existing behavior remains green;
remaining Linux/systemd/provider evidence is explicitly listed as pending.
**Risks and mitigations:** an old process or abandoned lease must fail closed rather than be guessed away; legacy
`/health` clients retain their route; authentication compatibility remains explicit; changes are limited to the
server boundary and do not alter Store formats.
**Validation:** `swift test --filter FountainStoreHTTPTests`, `swift test --filter FountainStoreSessionKitTests`,
`swift build --product FountainStoreHTTPServer`, OpenAPI parse/grep checks, `git diff --check`, and secret-leak scan.
## Cross-process Store change feed (2026-08-16)
**Goal:** make FountainStore's public `changes()` API usable by separate Reframe entities while preserving one
`StoreChange` vocabulary and the existing after-fsync/readable ordering.
**Scope:** `Sources/FountainStore/ChangeFeed.swift`, the commit publisher in `Store.swift`, focused change-feed tests,
and the public Store API documentation. The bridge carries only collection, encoded id, sequence, and put/delete
kind; subscribers still read values through the normal Store API.
**Non-goals:** no WAL parsing, no filesystem polling/watchers, no HTTP dependency, and no second event schema. The
macOS distributed notification is a delivery wake-up; FountainStore remains the behavioural authority and a missed
wake-up is handled by the subscriber's normal snapshot/reconciliation policy.
**Acceptance:** one process receives the existing stream; a second process receives the same `StoreChange` identity;
notifications are emitted only after WAL sync and in-memory apply; termination removes both observers; focused tests
and the Reframe StoreDump build pass.
## Chapter 55 — shared store authority (2026-08-10)
Objective: add a generic, documented Swift authority for explicit FountainStore selection, identity, readiness,
leases, recovery, and receipts, without introducing Reframe-specific semantics or changing the storage format.
Scope: new public session/authority API, deterministic state classification, owner-safe leases, tests, DocC/docs,
version bump, and Reframe integration revision.
Constraints: pure Swift plus Foundation; preserve the existing single-writer actor, ACID/WAL, manifest, crash
recovery, and stable `FountainStore` API; no filesystem search or HTTP dependency; no destructive cleanup.
Phases: governance first; generic API and tests; documentation and version release; Reframe migration; cross-repo
validation and synchronized provenance.
Risks and mitigations: path aliases are canonicalized before identity; incomplete stores are classified rather than
silently initialized; leases use atomic directory creation and token-checked cleanup; crash/relaunch tests cover
receipts and stale-owner recovery; Reframe consumes the released revision rather than a moving branch.
Validation: `swift test`, package build, public API documentation checks, Reframe dependency coherence, Reframe
focused launcher tests, fresh/resume/duplicate-store acceptance, and clean worktrees in both repositories.
## Linux staging contract slice (2026-08-23)
**Objective:** add the reproducible, non-active Linux installation contract needed before staging the typed server
authority on a remote host.
**Scope:** dedicated systemd unit, non-secret environment template, isolated release-directory convention, and
staging acceptance requirements. No active service switch, DNS/Caddy change, credential placement, or production
release is included.
**Validation:** macOS server tests/build remain green; Linux release build and artifact digest are required before
remote staging; the staged candidate must pass live/ready/authenticated-read/restart/no-secret checks.
## Linux evidence status after main push (2026-08-23)
Commit `f47fcb4` was pushed to `origin/main` after 109 local tests, the release server build, and diff checks passed.
The CI and Benchmarks workflows were triggered but not started because GitHub reports an account billing/spending-limit
failure. At the time of this entry no Linux executable or artifact digest existed; the later live staging acceptance
entry below supersedes that blocker for this candidate.
## Live staging acceptance — 0.4.0 candidate (2026-08-23)
The missing Linux prerequisite was provisioned on the verified staging host `book.fountain.coach` (`ubuntu-2gb-hel1-1`)
without changing the active Caddy publication services or the inactive production FountainStore unit.
- **Source:** `Fountain-Store@40f67b18bd8062a50494c1b6ddbd6d70da585f17`; source archive SHA-256
`b955fcf0373a94bd250a33195f8ca85704d75badd4f593e730e4fd1f94751d51`.
- **Toolchain:** official Swift 6.3.3 Ubuntu 24.04 x86_64 archive; Swift detached signature verified before
extraction. The private `swift-secretstore` build input was transferred separately at revision
`967af7d00a6478050bed4e86cad7aaadcc5a4ea6`; the staging-only manifest overlay is not a source release.
- **Executable:** `/srv/fountain-store/releases/0.4.0-staging-40f67b1/fountain-store-server`, SHA-256
`a556492ab9a56da463c72f04f6e6756646fd3af5dfa92ee8f473ad15789b1cd4`.
- **Runtime:** isolated loopback staging on `127.0.0.1:8790`, data path `/srv/fountain-store/staging-data/40f67b1`,
source revision passed through the runtime environment, and ephemeral authentication with no persisted secret file.
- **Read-only acceptance:** `/health/live` 200, `/health/ready` 200 with the same remote identity
`73608a53-b857-4ac4-9d9d-feaed4a041b0`; unauthenticated health 401; `/status` 200 with sequence 0;
`/collections` 200 empty; `/metrics?format=prometheus` 200 with all mutation counters at 0.
- **Boundary:** the process is not public, not Caddy-routed, and not an active systemd service. No remote Store
application writes, TLS issuance, provider mutation, production deployment, or security review is claimed.
## Governance read — supervised Chapter 95 staging (2026-08-23)
- **Chapters read** — 07 (bounded slice, focused validation, and evidence before claims); 08 (process, Store, and
release evidence are separate authorities); 95 (typed readiness, owner-safe lease, SecretStore custody, supervised
shutdown/restart, and systemd installation evidence).
- **What they forbid here** — treating the existing manually launched loopback process as supervised authority,
routing the staging candidate through Caddy, placing credentials in the repository, or calling a clean build a
supported Linux release.
- **Conflicts** — none. The requested staging-service work is narrower than production activation and preserves the
chapter's separate reverse-proxy, TLS, provider, and security-review boundaries.
- **Excluded, and why** — public routing, production unit activation, TLS, provider mutation, remote Store write
acceptance, and independent security review remain separate gates.
## Supervised staging lifecycle acceptance — 2f2f983 (2026-08-23)
The Chapter 95 supervised service slice is now accepted on the isolated staging host. This is a new candidate and
fresh managed Store path; earlier failed paths and stale-lease evidence were preserved rather than reused.
- **Source provenance:** `Fountain-Store@2f2f983`; source archive SHA-256
`998028e51d337922b028f84992b0a00e61fe0933882020a9c4a7ddc483834537`.
- **Executable:** `/srv/fountain-store/releases/0.4.0-staging-2f2f983/fountain-store-server`, SHA-256
`36145d7a1daeb915cf302a0cbab9527f6c274238846977c7c9aaa4bf2e7b5be7`; `/srv/fountain-store/staging/current`
points to that release.
- **Supervision:** `/etc/systemd/system/fountain-store-staging.service`, dedicated
`fountain-store-staging` user, loopback-only `127.0.0.1:8790`, SecretStore-backed FileKeystore, and no Caddy
or public listener change.
- **Readiness:** authenticated `/health/live`, `/health/ready`, `/status`, `/collections`, and `/metrics` each
returned HTTP 200. The running Store identity remained stable across systemd restart and clean stop/start.
- **Lease proof:** the owner lease was present while running; systemd stop completed immediately, the journal recorded
`shutdown lease=released`, and the owner lease was absent afterward. Final systemd state is active.
- **Authentication note:** health probes currently return 200 without a key by route design; authenticated probes were
also verified. This remains a contract/documentation choice to review separately, not a claim of public exposure.
- **Still open:** bounded record round-trip, backup/restore and doctor evidence, formal v0.4.0 release admission,
provider-specific Chapter 94 authorization, TLS/ACME Chapter 96 work, and independent security review.
## Staging persistence and backup acceptance — 2f2f983 (2026-08-23)
The isolated staging Store completed a bounded application-write and backup/restore exercise using the dedicated
acceptance collection `staging_acceptance_2f2f983`.
- **Record round-trip:** collection creation returned 201; record PUT returned 200 after the initial seed; read-back
returned `before-restore`.
- **Backup:** backup creation returned 201 and produced one listed backup. A post-backup mutation returned 200;
restore returned 202.
- **Restore boundary:** the in-process read immediately after the 202 restore still observed the post-backup mutation;
after the supervised restart, the restored value was `before-restore`. The restored value then survived the restart.
This establishes durable restore/reopen behavior, not instantaneous in-process cache invalidation.
- **Runtime persistence:** final supervised service state is active; the Store reopened with checksum verification
enabled, and the record remained readable after restart. The active listener is `127.0.0.1:8790`; the deployed
release symlink resolves to `/srv/fountain-store/releases/0.4.0-staging-2f2f983` with binary SHA-256
`36145d7a1daeb915cf302a0cbab9527f6c274238846977c7c9aaa4bf2e7b5be7`.
- **Doctor boundary at that time:** no standalone doctor command was present; reopen/checksum verification and the
HTTP backup/restore proof were recorded as the temporary evidence. The dedicated doctor/repair tool was added in
commit `6b69144` below.
## Release-admission audit — 2026-08-23
- Full local Swift suite: **109 tests, 0 failures**.
- `swift build --product FountainStoreHTTPServer`: **PASS**.
- OpenAPI presence checks for live/ready, collections, and backups: **PASS**.
- `git diff --check` and worktree cleanliness: **PASS**; `main` is synchronized with `origin/main` at the evidence
record commit.
- The staging unit remains isolated and active after the audit. No production unit, Caddy configuration, DNS, TLS,
provider state, or public route was changed.
Formal v0.4.0 release admission is intentionally not asserted: the provider authorization, TLS/ACME lifecycle, and
independent security review gates are external or separately governed.
## Doctor tooling — 6b69144 (2026-08-23)
The missing doctor/repair tool is implemented and verified locally and on Linux staging.
- **Tool:** `FountainStoreDoctor --path PATH [--json] [--repair-stale-lease]`.
- **Safety boundary:** inspection uses one explicit path, validates authority manifest/receipt consistency, reports
lease owner state, and opens the Store with checksum verification enabled. It never searches for a Store or acquires
an application lease.
- **Repair boundary:** stale-lease repair is explicit and refuses live or malformed owners; it checks the owner file
has not changed before token-independent stale cleanup. Normal owners still release through their token-matched
`FountainStoreLease`.
- **Validation:** full local suite now passes **111 tests, 0 failures**; the doctor product builds locally and with
Swift 6.3.3 on Ubuntu 24.04 x86_64.
- **Remote staging:** `/srv/fountain-store/tools/6b69144/FountainStoreDoctor`, SHA-256
`a2584a162366a71e1cd165ae894895813450f90210cd624a662141f1c26f4527`. Read-only doctor exit 0 reported ready
authority, live owner PID 18899, integrity pass, sequence 2, and one backup for the managed staging Store.
- **Provenance:** source archive for `6b69144` SHA-256
`cf8b4abf126bed0e9e7dee459978afeba9004e18706307eb79e1b04f37e4a99a`.
The formal release gate is narrowed but not closed: a provider-specific authorization adapter/evidence, TLS/ACME
lifecycle, and independent external security review remain outside this repository's self-verifiable tooling.
## Restart-on-failure staging probe — stale lease finding (2026-08-23)
- A bounded `SIGKILL` of only `fountain-store-staging.service` proved systemd scheduling: the unit was restarted, but
the candidate refused startup with `FountainStoreSelectionError.leased` because the crashed owner lease remained.
- The released doctor reported the explicit Store path `/srv/fountain-store/staging-data/final-2f2f983` as
`leaseOwnerState=stale`, with integrity pass, receipt ready, source revision `2f2f983`, and two backups.
- After the unit was stopped, `FountainStoreDoctor --repair-stale-lease` removed only that stale owner. The same
service then returned ready on loopback with the existing identity, record, and integrity intact; the doctor reported
the new live owner and exit 0.
- **Gate status:** systemd restart scheduling is observed, but automatic application recovery after an unclean crash is
not accepted. The next implementation slice is a reviewed crash-safe service/lease recovery path; it must preserve
refusal of live owners and must be tested before any production-shaped activation is considered.
## When to Use
Use this protocol for multi-step, high-risk, or cross-cutting changes.
## Plan Template
Include the following sections in your plan:
- Objective: what outcome is intended.
- Scope: what is in and out.
- Constraints: non-negotiable rules from `AGENTS.md`.
- Phases: ordered phases with acceptance criteria for each.
- Risks and mitigations: what could go wrong and how to avoid it.
- Validation: how to confirm the result.
## Operating Rules
- Keep plans minimal and focused on intent, not step-by-step tooling.
- Update the plan as phases complete.
- If uncertainty is high, surface assumptions early.
## Planned deployment — `store.fountain.coach` owner-controlled operational production (2026-08-23)
**Objective:** promote FountainStore from the accepted Linux staging candidate to owner-controlled operational
production at the exact tuple `store.fountain.coach → 65.109.14.71`, without claiming independent security review.
### Unlock sequence
1. **Admit the release candidate**
- Build from a clean pinned source revision and committed `Package.resolved`.
- Run the full test suite and Linux release build.
- Record semantic version, source revision, dependency revisions, executable SHA-256, and package-resolution digest.
- Sign the release manifest and executable with the owner-controlled release key; retain the previous release for rollback.
2. **Create the host production identity**
- Create the dedicated `fountain-store` user/group and `/var/lib/fountain-store` data root.
- Install the verified binary only at the exact production release path.
- Install the reviewed `fountain-store.service` template with production-safe defaults and no staging lease-recovery flag.
3. **Admit host-owned credentials**
- Create or resolve a production-only `FS_API_KEY` through the host SecretStore/FileKeystore.
- Store only the protected reference in `/etc/fountain-store/fountain-store.env`.
- Verify authenticated refusal/success without printing or persisting the secret.
4. **Record provider and change authority**
- Record the exact host/provider identity, target hostname, scope, expiry, consent, revocation path, maintenance
window, rollback release, and post-change verification plan in a redacted authorization receipt.
- Keep the external security review as a later assurance lane; it is not part of this operational activation.
5. **Complete Caddy-independent TLS and public routing**
- Create exactly `store.fountain.coach A 65.109.14.71` with the reviewed TTL.
- Add native TLS support to `FountainStoreHTTPServer` using a reviewed SwiftNIO TLS boundary; the application owns
the 443 listener and redirects or refuses cleartext on the 80 listener according to the approved policy.
- Use host-owned SecretStore references for the certificate/key and ACME account/order state; no private key enters
the repository, unit arguments, logs, or Store receipts.
- Complete the provider-neutral ACME/HTTP-01 handoff, verify certificate coverage and expiry, and remove the
deployment's dependency on the Caddy host block.
6. **Activate and accept**
- Stage the release, verify signature/digest, atomically switch the production release pointer, and start the unit.
- Verify `/health/live`, `/health/ready`, authenticated read/write round trip, stable Store identity, no-secret logs,
native HTTPS, certificate renewal metadata, and the exact public hostname.
- Record source, artifact digest, service PID, Store path/identity, hostname, certificate, and terminal receipts.
7. **Prove rollback and handoff**
- Exercise the owner-safe rollback path against the retained previous release in a bounded maintenance window.
- Verify readiness, Store identity, authentication, and HTTPS after rollback and restoration.
- Record the final active release and the revocation procedure.
**Completion definition:** operational production is established only when steps 1–7 have redacted evidence bound to
the same source revision, signed artifact, host, service, Store, hostname, and certificate. The result may claim
owner-controlled operational use; it must not claim independent security review or certification.
## Native TLS/ACME staging acceptance (2026-08-23)
The native edge slice is accepted on the staging host, without changing public production delivery.
- Built `FountainStoreHTTPServer` with native SwiftNIO TLS and deployed the committed staging artifact.
- Issued a real Let’s Encrypt staging certificate for `store.fountain.coach` through the host-custodied ACME account and separate certificate key in SecretStore.
- Served HTTP-01 through the native FountainStore listener on port 80 while Caddy was stopped; the CA completed issuance.
- Verified the certificate/key pair with OpenSSL, native HTTPS `/health` returned `200`, and cleartext returned the configured `308` HTTPS redirect.
- Corrected the Linux SEC1 PEM handoff in `Fountain-Coach/SwiftACMEKit` and pinned revision `ddcfa56` here.
- Restored Caddy and the normal loopback staging service after acceptance. No production DNS, Caddy route, certificate, provider mutation, or external security-review claim is made.
## Planned extension — native TLS/ACME edge for Caddy-independent production (2026-08-23)
**Objective:** extend `FountainStoreHTTPServer` into a self-contained public HTTPS edge for
`store.fountain.coach`, while keeping the existing Caddy route unchanged until native acceptance is complete.
### Implementation slices
1. **TLS transport boundary**
- Add the reviewed SwiftNIO TLS dependency and configuration types.
- Load certificate/key references only from host-owned SecretStore/FileKeystore paths.
- Reject missing, unreadable, expired, or mismatched certificate material before readiness.
- Keep private key bytes out of arguments, logs, telemetry, Store records, and repository files.
2. **Native listener and redirect policy**
- Add explicit production listener configuration for ports 443 and 80.
- Serve HTTPS directly on 443 and implement the approved cleartext redirect/challenge policy on 80.
- Preserve loopback-only staging defaults and prevent accidental public binding in staging.
- Add systemd capability/privilege settings required for binding 443 without broad root execution.
3. **ACME lifecycle**
- Integrate the provider-neutral ACME boundary for account, order, nonce, challenge, certificate, renewal, and revocation state.
- Keep ACME account keys and certificate material in host SecretStore custody.
- Support HTTP-01 challenge serving, renewal-window scheduling, atomic replacement, and rollback to the previous certificate.
- Record only redacted order/certificate fingerprints, expiry, and terminal outcomes.
4. **Edge protections**
- Add bounded request body/header sizes, connection and request timeouts, slow-client protection, rate limits, and bounded concurrency.
- Preserve authenticated API behavior and ensure health endpoints cannot bypass required production authentication where policy requires it.
- Add redacted access/security telemetry with no credentials or sensitive payloads.
5. **Acceptance and migration**
- Add deterministic unit tests for TLS configuration, certificate failure, renewal, challenge, reload, rollback, limits, and cleartext policy.
- Run isolated native HTTPS acceptance on a distinct port and host, then live acceptance on the exact production tuple.
- Record certificate coverage/expiry, source revision, artifact digest, service identity, Store identity, and rollback evidence.
- Remove the Caddy `store.fountain.coach` route only after native HTTPS is accepted; retain a recoverable Caddy rollback configuration until post-cutover verification completes.
**Non-goals:** no certificate/private-key commit, no shared Caddy changes before native acceptance, no production DNS cutover before the native edge passes, and no independent-security-review claim.
### Kit binding progress (2026-08-23)
- Created and pushed the organization-owned private repository `Fountain-Coach/SwiftACMEKit` on `main`.
- Bound FountainStore to pinned Kit revision `0c50d35` through the `ACMECore` product.
- The Kit is a tested scaffold candidate: it does not claim real certificate issuance, production account custody,
host binding, or certificate deployment.
## Planned extension — native estate routing and static publication (2026-08-23)
**Objective:** make FountainStoreHTTPServer capable of serving the existing Fountain Coach static publication estate
directly, with deterministic host-based routing and traversal-safe static files, so those hosts can later migrate away
from Caddy one at a time.
### Scope
- In scope: explicit host route configuration, exact-host matching, static roots, index files, content types, cache
headers, conditional requests, range refusal, traversal protection, symlink/root escape protection, and a safe
default response for unknown hosts.
- Out of scope for this slice: reverse proxying to Composer/Image Cloud, DNS changes, Caddy changes, TLS cutover,
certificate issuance, and public production claims.
### Acceptance
1. Configuration rejects malformed routes, duplicate hosts, non-directory roots, and unsafe paths.
2. Static requests are served only for configured hosts, with no path escape and no accidental directory listing.
3. API routes remain unchanged and host routing cannot bypass API authentication.
4. Tests cover host matching, traversal, MIME types, cache/conditional behavior, and unknown-host refusal.
5. The Linux release artifact builds from the committed source and is exercised in isolated staging before any live
host cutover.
### Upstream routing slice (2026-08-23)
- Added explicit host/path-prefix upstream routes for native HTTP/1.1 proxying.
- Added request-body preservation, hop-by-hop header filtering, upstream host forwarding, bounded response collection,
and 502 failure behavior.
- Added the current Library route map as a reviewed deployment example; no live route is enabled by this commit.
- Acceptance: 29 FountainStore HTTP tests pass. Live upstream acceptance remains staging-only and must exercise the
actual Composer/Image Cloud services before any Caddy route is changed.
- Staging acceptance completed on host release `19dc268`: static `fountain.coach` and `governance.fountain.coach`
returned 200, an unknown host returned 404, and the Library `/v1/assets` route propagated the intended upstream
401. The service stayed loopback-only; Caddy and public traffic were unchanged.
## Chapter 97 host-agent service boundary (2026-08-24)
Added `FountainStoreHostAgentService` to the `FountainStoreHTTP` library. The service binds operations to the exact
host target and identity, verifies an injected scoped credential, requires ready server authority, enforces expiry
and idempotency, checks the install artifact digest, and refuses mutations until an explicit operation admission is
provided. Revoke is terminal and receipts contain only redacted identity/readiness evidence.
This is the host-side service boundary only. Provider provisioning, SecretStore retrieval, TLS handoff, HTTP route
exposure, systemd installation, and real install/rollback executors remain separate slices. No host or provider was
contacted.
Validation: `swift test --package-path . --filter FountainStoreHTTPTests` passed 32 tests and
`swift build --package-path . --product FountainStoreHTTPServer` passed. Existing compiler warnings remain
unrelated to this slice.
The standalone `FountainStoreHostEnrollmentKit` is now published by Fountain-Coach at tag `0.1.0`, but its
CryptoKit signing API currently declares Apple platform availability. FountainStore's Linux server therefore does
not consume it yet. The kit portability slice is now at tag `0.1.2`, using `swift-crypto` and the existing
FountainStore deployment floor. FountainStoreHTTP pins that exact release through an explicit adapter. This binds
the wire contract only; no route is enabled, no credential lookup is performed, and no remote host has been
contacted by this commit.
### Protected host-agent route slice (2026-08-24)
Added the disabled-by-default `/agent/v1/{status,install,rollback,rotate,revoke}` POST route family. Requests use
the released kit contract, exact operation-path matching, base64 credential transport in a dedicated header, and
redacted kit receipts. The route delegates to `FountainStoreHostAgentService`; it does not perform provider,
filesystem, certificate, or install work. Enabling it requires TLS, exact target/identity/artifact/fingerprint
configuration, and a SecretStore account. Mutation admission remains empty unless explicitly configured.
Validation: the 32-test `FountainStoreHTTPTests` suite and `FountainStoreHTTPServer` build pass. No route was enabled
and no host or provider was contacted.