# CI sandbox v2: subordinate id mapping (contract between workers)

**Goal:** jobs get a real range of uids and gids, so package managers that `chown` to other users work (`apt`, `apk add`, `useradd`, …). Container uid 0 maps to the runner's uid, and container ids `1..=N` map to the runner's `/etc/subuid` and `/etc/subgid` range. This uses the setuid helpers `newuidmap` and `newgidmap` from shadow-utils, if present. Without them, the runner falls back to the v1 single-uid mapping.

The global rules in `~/.cache/forge-orch/waveb-common.md` apply.

The existing sandbox is described in `docs/ci-builtin-executor.md` and `~/.cache/forge-orch/libcontainer-notes.md`. Read both.

## Host facts
- `/usr/bin/newuidmap` and `/usr/bin/newgidmap` exist.
- The subuid range is `tahinli:524288:65536`.
- `unshare --map-auto --map-root-user chown 1000:1000 f` works and gives host owner `525287`. That is `start + 999`, because container 0 is the runner and container 1 is `start`.
- Tests can use this command to create files with a subordinate owner. Skip the test when it fails.

## Mapping
- **subuid mode:** container `0` maps to the runner uid (count 1), and container `1` maps to `start` (count `n`), where `n = min(count, 65536)`. The same applies to gids. Container size is `1 + n`.
- **single mode:** today's behaviour, `0 → uid`, count 1.

## Worker V-A: `crates/forge-runner/src/sandbox/idmap.rs`, plus the hidden-command dispatch
```rust
#[derive(Clone, Copy, Debug, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum UsernsMode { #[default] Auto, Single, Subuid }

#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
pub struct IdMap { pub uid: u32, pub gid: u32, pub sub_uid: Option<(u32, u32)>, pub sub_gid: Option<(u32, u32)> } // (start, n)
impl IdMap {
    pub fn single() -> Self;                                      // current uid/gid, no ranges
    pub fn resolve(mode: UsernsMode) -> Result<Self, IdMapError>; // see below
    pub fn is_subuid(&self) -> bool;
    pub fn size(&self) -> u32;                                    // 1 + n, or 1
    pub fn describe(&self) -> String;                             // "subuid 524288+65536" | "single uid 1000"
}
impl Default for IdMap  // = single()
pub fn parse_subid(content: &str, user: &str, uid: u32) -> Vec<(u32, u32)>; // match name OR numeric uid, skip comments/bad lines
/// SINGLE-THREADED processes only. unshare(NEWUSER|NEWNS), install the maps, make "/" rec-private.
pub fn enter_mapped_userns(map: &IdMap) -> Result<(), IdMapError>;
/// `<exe> __sandbox-rm <path>`, IdMap JSON on stdin: enter the userns, give dirs u+rwx, remove_dir_all. Exit 0, or 125 + "forge-sandbox: " line.
pub fn rm_helper_main(args: &[OsString]) -> i32;
/// Blocking. Single mode: remove_dir_all with a chmod-u+rwx retry, no spawn. Subuid mode: run `<exe> __sandbox-rm <path>` with the map on stdin.
/// Missing path = Ok.
pub fn remove_tree(map: &IdMap, path: &Path, exe: &Path) -> io::Result<()>;
```

**`resolve`**
- **Auto:** use subuid when all of these hold:
  - `newuidmap` and `newgidmap` exist in `/usr/bin`, `/bin`, `/usr/sbin` or `/sbin`;
  - `/etc/subuid` and `/etc/subgid` each have an entry for the user (by name, from `getpwuid`, or by numeric uid);
  - that entry has `count >= 65536`. Take the first such range. A smaller range is accepted with a warning only if it is ≥ 1000.

  Otherwise use single.
- **Subuid:** return an error instead of falling back.
- **Single:** `single()`.

**`enter_mapped_userns`, subuid mode**
1. Before unsharing, create a pipe and fork a child. The child blocks reading the pipe.
2. The parent runs `unshare(CLONE_NEWUSER | CLONE_NEWNS)`, then writes one byte.
3. The child runs these commands:
   - `newuidmap <parent pid> 0 <uid> 1 1 <start> <n>`
   - `newgidmap <parent pid> 0 <gid> 1 1 <start> <n>`

   It runs them via `std::process::Command` (the child is single-threaded, so this is allowed), then `_exit`s with 0 or 1.
4. The parent waits with `waitpid` and checks the status. On failure it returns an error including newuidmap's stderr.
5. **Do NOT write `setgroups deny`**, because newgidmap needs `allow`. After mapping, call `setgroups(&[])`. Then make the mounts rec-private as today.

**Single mode** keeps today's code: `setgroups deny` and direct map writes. Move it here from `helper.rs`; the helper then calls `enter_mapped_userns`.

**Dispatch:**
- `cli::is_sandbox_invocation()` becomes true for any argv[1] starting with `__sandbox`.
- Add `pub fn hidden_main() -> i32` in `cli.rs`, which routes:
  - `__sandbox` → `sandbox::helper_main()`
  - `__sandbox-rm` → `idmap::rm_helper_main`
  - `__sandbox-unpack` → `crate::oci::unpack_helper_main()`. This belongs to V-C; if it doesn't exist yet, add a stub returning 125 in `oci/mod.rs` that V-C replaces.
- Use `hidden_main` from `cli::main_with_args`, `forge-runner`'s `main.rs` and forge-web's `main.rs`.

**Tests**
- `parse_subid` cases.
- `size` and `describe`.
- `resolve(Auto)` on this host yields subuid.
- An integration test in `tests/sandbox.rs` (or a new `tests/idmap.rs`) that runs `forge-runner __sandbox-rm` on a TempDir holding a file chowned to 1000 by `unshare --map-auto --map-root-user`. The dir must be gone afterwards. Skip when `unshare --map-auto` fails.

## Worker V-C: images with real ownership, in `crates/forge-runner/src/oci/`
- **`unpack_layers` gains a `ownership: Ownership` parameter:**
  - `pub enum Ownership { Flatten, Preserve { size: u32 } }`.
  - `Flatten` is today's behaviour.
  - `Preserve`:
    - `lchown` every entry, including symlinks, to the tar header uid and gid.
    - Ids `>= size` become 0 and are counted in `UnpackStats.remapped_ids`.
    - setuid and setgid are still stripped.
    - Directories get their exact mode at the end, deepest first, instead of `mode | 0o700`.
  - Update all callers and tests.
- **`ImageStore::with_idmap(map: IdMap, helper_exe: PathBuf)`:**
  - When the map is subuid, the rootfs dir is `rootfs/<digest>-s<uid start>-<n>`, so a changed range re-unpacks. Single keeps `rootfs/<digest>`.
  - Unpacking in subuid mode runs `<helper_exe> __sandbox-unpack` from `spawn_blocking`. The JSON `UnpackRequest { map, rootfs, layers, max_bytes }` goes on stdin and `UnpackStats` JSON comes back on stdout.
  - `pub fn unpack_helper_main() -> i32` (in `oci/mod.rs` or a new `oci/helper.rs`):
    - reads the request;
    - calls `idmap::enter_mapped_userns`;
    - calls `unpack_layers(.., Ownership::Preserve { size: map.size() })`;
    - prints the stats and exits 0, or 125 with a "forge-sandbox:" line.
- **GC and staging cleanup** remove trees with `idmap::remove_tree(&map, path, &helper_exe)`, because subuid-owned files can't be deleted by the runner directly.
- **Tests:**
  - `Preserve` unit tests that only check the remapping logic (a pure function, `fn map_id(id, size) -> (u32, bool)`).
  - Path naming per mode.
  - An integration test that unpacks a crafted layer with a file owned by 1000:1000 through `forge-runner __sandbox-unpack`, then checks that the host owner is `start + 999`. Skip when subuid is unavailable.

## Worker V-B, after V-A: sandbox, executor, config, check, docs
- **`SandboxRequest`** gains `#[serde(default)] pub idmap: IdMap`. The helper calls `idmap::enter_mapped_userns(&req.idmap)`. The container spec's uid and gid mappings become a single line each, `0 0 <size>`, which is relative to the helper's namespace.
- **Workspace ownership:** after the job ends, the helper (still root in the userns) walks `req.workspace` without following symlinks and `lchown`s every entry whose uid or gid is not 0 to 0:0. The host runner then owns everything for artifacts and cleanup.
- **Executor:** `reap_job` and any job-dir cleanup use `idmap::remove_tree`. `BuiltinExecutor` gets the map.
- **Config:** `[sandbox] userns = "auto" | "single" | "subuid"` (default auto). It is resolved once at startup, logs `describe()`, and is passed to `ImageStore::with_idmap` and `BuiltinExecutor`. A failure to resolve in subuid mode is a startup error.
- **`forge-runner check`:** prints the mode. In subuid mode it also checks that a probe job can `chown 1000:1000` a file in `/tmp`.
- **docs/ci.md:**
  - replace the single-uid caveat with the v2 behaviour;
  - document the requirements: shadow-utils `newuidmap`/`newgidmap` and an `/etc/subuid` and `/etc/subgid` entry (`usermod --add-subuids 100000-165535 --add-subgids 100000-165535 <runner user>`);
  - document the fallback;
  - explain that cached image files on the host are owned by subordinate ids.
- **Tests (`tests/sandbox.rs`, skip when subuid is unavailable):**
  - a job that does `touch /tmp/f && chown 1000:1000 /tmp/f && stat -c '%u %g' /tmp/f` prints `1000 1000`;
  - `cat /proc/self/uid_map` has size `1 + n`;
  - a job that creates `/workspace/a` and chowns it to 1000 leaves `a` owned by the runner uid on the host;
  - the job dir is gone afterwards;
  - all existing single-mode tests still pass, with the map forced to single.