p4n4-docs

ADR-002: Per-Layer Subdirectories in Multi-Layer Projects

Date: 2026-07 Status: Accepted Deciders: raisga core team

Context

p4n4 init --layer iot,ai originally copied every enabled stack’s files into the project root. The stacks collide there:

• Both ship a docker-compose.yml, and both ship a scripts/selector.sh with different contents — the second copy silently overwrote (or crashed on) the first.
• Each layer’s .env write clobbered the previous layer’s secrets.
• The compose files cannot be merged into one Compose project: p4n4-iot defines the p4n4-net network (bridge driver, ipam subnet) while p4n4-ai and p4n4-edge declare the same network external: true. Compose rejects a merged model with both definitions, and include: fails on the conflict as well.

The stacks are deliberately designed to run as separate Compose projects sharing one external network (see ADR-001), so the scaffolded project has to preserve that.

Decision

Multi-layer projects scaffold each layer into its own subdirectory:

my-project/
├── .p4n4.json          ← manifest at the root, lists all layers
├── iot/                ← docker-compose.yml, config/, scripts/, .env
└── ai/                 ← docker-compose.yml, config/, scripts/, .env

Single-layer projects keep the original flat layout (stack files at the project root). Layout detection is derived, not recorded: a docker-compose.yml at the project root means flat; otherwise each <layer>/docker-compose.yml is a stack directory. The rules live in p4n4_lib.layout so all clients (CLI, API, dashboard) resolve projects the same way.

Consequences for the CLI commands:

• p4n4 up [STACK] / p4n4 down [STACK] run per stack directory, in dependency order iot → ai → edge (reversed for down); the optional stack argument filters to one.
• p4n4 logs requires --stack <name> (or --no-follow) in multi-layer projects.
• p4n4 validate checks each layer’s required files and .env inside its directory.
• Cross-stack .env keys (INFLUXDB_TOKEN, INFLUXDB_ORG, INFLUXDB_BUCKET) are written identically to every layer at init, and p4n4 secret rotate generates one value per key and updates every .env containing it.

Consequences

Positive:

• Stack files are byte-for-byte what the stack repos ship — no YAML merging or rewriting.
• Each stack keeps its standalone semantics: docker compose works directly inside a layer directory, and p4n4 add / p4n4 remove map to creating/deleting a directory.
• The network ownership model from ADR-001 (iot owns p4n4-net, others attach) is preserved exactly.

Negative:

• Shared secrets are duplicated across layer .env files; consistency depends on the CLI (init and secret rotate) rather than a single file. Manual edits must be repeated per layer.
• Commands that operate “on the project” iterate several Compose projects, so failures can leave some stacks up and others down (mitigated by dependency ordering).

Alternatives considered

1. Merge compose files into one project (COMPOSE_FILE=iot.yml:ai.yml or a root file with include:) — rejected: the conflicting p4n4-net definitions error out, and working around it means rewriting the stacks’ YAML at scaffold time.
2. Single shared .env at the project root — rejected with the per-directory Compose model: compose only auto-loads the .env next to the compose file, and passing --env-file ../.env everywhere breaks standalone docker compose usage inside a layer directory.

References

• ADR-001 — multi-repository architecture and network ownership
• Architecture reference §5 — network & secrets strategy
• CLI reference