# Forge Architecture Status: milestones 1 and 2 are implemented. Later milestones are outlined here and not built yet. "Forge" is a placeholder name. Renaming is a find-and-replace. Sources: the M1 plan (`~/.claude/plans/purring-prancing-wirth.md`), the interface contract (`docs/m1-interfaces.md`), and the research notes (`docs/research.md`, sections 1-16). Items marked *(unverified)* in the research are not confirmed. --- ## 1. Goals and principles - **Independence.** Forge is a standalone app. It has no dependency on other projects, no shared code, and no hosted forge in the loop. The owner was banned from GitHub and lost five years of work, so nothing critical may depend on a third party's account decisions. - **Data safety first.** Metadata and repositories both have to survive disk loss, a compromised server, and a provider account termination. Section 8 covers this. - **Single writer.** One primary instance accepts writes. Replicas and backups are copies. Failover is manual and fenced (section 8.5). There is no consensus layer. - **The DB holds metadata.** Users, sessions, keys, tokens, repos, and later issues, PRs, labels, and milestones live in Turso. Issues and PRs are not stored in git. - **Git is the source of truth for code.** Objects and refs live in bare repositories on disk. The DB never holds file contents or ref values that git can answer. - **Latest dependencies.** Every crate is added with `cargo add` at its latest release and recorded in `[workspace.dependencies]`. Versions are not copied from other projects. - **Security by default.** Deny unless allowed. One authorization function. No shell. Private repos return 404. Secrets are hashed or sealed at rest. Section 9 has the checklist. --- ## 2. Stack and why | Layer | Choice | Why | |---|---|---| | Web and HTTP | Topcoat 0.10 on hyper | Server-rendered UI and git smart-HTTP in the same app. Topcoat routes accept streaming bodies with no size limit and no default request timeout, which git needs. | | Git smart-HTTP | Routes in Topcoat, backed by the `git` binary | No axum. The git child process streams request and response bodies directly. | | Metadata DB | Turso | Embedded SQLite-compatible store. Holds all metadata. WAL with `synchronous=FULL` (section 8.3). | | Git serving | `git` binary (`upload-pack`, `receive-pack`, merges, blame, verify, maintenance) | Git's own implementation is the only trustworthy server side. | | Git reads | `gix` in-process | Object, tree, and blob reads, rev-walk, diffs, rename detection, archives (research §7). | | SSH | russh 0.64.x | Built-in SSH server for git over SSH. No OpenSSH dependency. | | Hooks | `forge hook` subcommand over a unix socket | Hook policy lives in the app, not in files users can edit. | Not used: `git2` (adds a C dependency, research §7). Dumb HTTP (disabled, research §1). --- ## 3. Crates and responsibilities Cargo workspace at the repo root. Edition 2024, resolver 3. ``` project_git/ Cargo.toml .gitignore config/forge.toml deploy/forge.service style/main.scss docs/{research.md, architecture.md, m1-interfaces.md} crates/ forge-core/ domain types, store, migrations, config, secret key, authz, names, passwords, tokens, sessions forge-git/ safe git command builder, process spawn and streaming, pkt-line, repo init, managed gitconfig, version check forge-ssh/ russh server: pubkey auth, exec allow-list and parse, GIT_PROTOCOL env, piping, timeouts forge-web/ crate, builds the `forge` binary: Topcoat UI, auth, smart-HTTP routes, internal hook API, `hook` subcommand, admin CLI ``` Module ownership (from the contract): - **forge-core** - `error.rs`, `time.rs` (`now_stamp()`, `stamp_after()`), `model.rs` (domain types) - `config.rs` (`Config::load`), `secret.rs` (`load_or_create_key`, `seal`, `open`) - `names.rs` (`normalize_name`, `validate_name`, `RESERVED`), `password.rs` (argon2id) - `token.rs` (`generate_token`, `hash_token`, `constant_time_eq`), `session.rs` - `authz.rs` (`Action`, `Denied`, `authorize`) - `store/mod.rs` (`Store`), `store/migrate.rs` (migration runner) - `migrations/0001_init.sql` - **forge-git** - `command.rs` (`GitCommand`), `version.rs`, `pktline.rs`, `repo.rs` (`repo_path`, `init_bare`, `list_branches`) - `setup.rs` (`write_managed_gitconfig`, `write_hook_shims`), `service.rs` (`GitService`, `service_command`) - **forge-ssh** - `backend.rs` (`GitBackend` trait), `parse.rs` (`parse_exec`), `server.rs` (`run`, host key load or create, russh `Handler`) - The crate does not know about the DB. It calls `GitBackend` for auth and for preparing the git process. - **forge-web** - `main.rs` (arg dispatch: `serve` default, `hook `, `admin create-user`), `boot.rs`, `state.rs` - `auth.rs` (session cookie, `current_user()`, CSRF check, Basic-auth parse, token auth) - `git_http.rs`, `internal.rs` (hook API on unix socket), `hook_cmd.rs`, `ssh_backend.rs` (impl `GitBackend`), `admin_cli.rs` - `views/mod.rs`, `pages/{setup, login, dashboard, repo, new_repo, settings_keys, settings_tokens, settings_sessions, admin_users}.rs` Domain types (`forge-core::model`): `Id = String` (ULID), `Visibility {Public, Private}`, `Role {Read, Triage, Write, Maintain, Admin}`, plus `User`, `Repo`, `SshKey`, `AccessToken`, `Session`. Store API (`Store`: async, `Clone`, wraps the Turso connection). `open()` sets WAL, `synchronous=FULL`, `foreign_keys=ON`, and a busy timeout, then runs migrations. It exposes user, session, SSH key, token, repo, collaborator, and push-event methods (the full list is in the contract). --- ## 4. Data model ### 4.1 Milestone 1 tables (`0001_init.sql`) Conventions: ids are ULID text. Timestamps are UTC RFC 3339 with 9 fractional digits and a `Z` suffix. Booleans are integers 0/1. Foreign keys are enforced. **`schema_migrations`** - `version` (PK). Records applied migrations. **`users`** - `id` TEXT PK (ULID) - `username` TEXT NOT NULL (display form) - `username_folded` TEXT NOT NULL UNIQUE (NFKC + case-fold) - `email` TEXT NULL - `password_hash` TEXT NOT NULL (argon2id) - `is_admin` INTEGER NOT NULL DEFAULT 0 - `created_at` TEXT NOT NULL **`sessions`** - `id` TEXT PK - `user_id` TEXT NOT NULL REFERENCES `users(id)` ON DELETE CASCADE - `token_hash` TEXT NOT NULL UNIQUE (sha256 of the random session token; the raw token is only in the cookie) - `csrf_token` TEXT NOT NULL - `created_at`, `expires_at`, `last_seen` TEXT NOT NULL - `user_agent` TEXT NULL - Queries return only unexpired rows. **`ssh_keys`** - `id` TEXT PK - `user_id` TEXT NOT NULL REFERENCES `users(id)` ON DELETE CASCADE - `title` TEXT NOT NULL - `algorithm` TEXT NOT NULL - `key_blob` BLOB NOT NULL UNIQUE (raw SSH wire-format public key; exact-match lookup) - `fingerprint` TEXT NOT NULL - `created_at` TEXT NOT NULL, `last_used_at` TEXT NULL **`access_tokens`** - `id` TEXT PK - `user_id` TEXT NOT NULL REFERENCES `users(id)` ON DELETE CASCADE - `name` TEXT NOT NULL - `token_hash` TEXT NOT NULL UNIQUE (hex sha256; the token is shown once) - `scopes` TEXT NOT NULL (serialized list) - `expires_at` TEXT NULL (default expiry is 30 days, research §10) - `created_at` TEXT NOT NULL, `last_used_at` TEXT NULL **`repos`** - `id` TEXT PK (ULID; determines the on-disk path) - `owner_id` TEXT NOT NULL REFERENCES `users(id)` - `name` TEXT NOT NULL (display form) - `name_folded` TEXT NOT NULL, with `UNIQUE(owner_id, name_folded)` - `description` TEXT NOT NULL DEFAULT '' - `visibility` TEXT NOT NULL CHECK (`public` or `private`) - `default_branch` TEXT NOT NULL - `archived` INTEGER NOT NULL DEFAULT 0 - `created_at` TEXT NOT NULL **`repo_redirects`** - Old owner and name mapped to a repo id, so old web and git URLs keep working after a rename. Exact columns are set in `0001_init.sql`; the contract does not fix them. **`collaborators`** - `repo_id` TEXT NOT NULL REFERENCES `repos(id)` ON DELETE CASCADE - `user_id` TEXT NOT NULL REFERENCES `users(id)` ON DELETE CASCADE - `role` TEXT NOT NULL CHECK (`read`, `triage`, `write`, `maintain`, `admin`) - PK `(repo_id, user_id)` **`push_events`** - `id` TEXT PK - `repo_id`, `actor_id` TEXT NOT NULL (foreign keys) - `ref_name`, `old_oid`, `new_oid` TEXT NOT NULL - `protocol` TEXT NOT NULL CHECK (`http` or `ssh`) - `created_at` TEXT NOT NULL Secrets (`forge.key` is not in the DB). Values sealed with XChaCha20-Poly1305 are stored as `v1:` plus base64(nonce || ciphertext). M1 has no sealed column yet. Mirror credentials and TOTP secrets are the first candidates (M5 and later). ### 4.2 Later tables (outline) | Table | Milestone | Purpose | |---|---|---| | `issues` | M3 | Number per repo, title, body, state, author, timestamps | | `comments` | M3 | Issue and PR comments, polymorphic by parent, edit history | | `labels` | M3 | Per-repo labels, plus an `issue_labels` join | | `milestones` | M3 | Per-repo milestones, due date, state | | `pulls` | M4 | PR number, base and head refs, head oid, merge state, outdated flag | | `reviews` | M4 | Reviewer, state (approve, request changes, comment), commit oid | | `review_comments` | M4 | Commit oid, path, line, side, diff hunk, outdated flag (kept, never deleted) | | `webhooks` | M5 | Target URL, events, encrypted secret (sealed) | | `deliveries` | M5 | Delivery id, attempt, status, request and response, retry state | | `mirrors` | M5 | Pull and push mirrors, interval, backoff, sealed credentials | | `commit_statuses` | M6 | Commit oid, context, state, target URL, creator | | `replication_state` | M5 | Per-repo, per-replica last ref checksum, last success and failure | --- ## 5. Request flows ### 5.1 Web page with session and CSRF 1. The browser sends a request with cookie `__Host-forge_session` (`Secure`, `HttpOnly`, `SameSite=Lax`, `Path=/`). 2. `auth::current_user()` hashes the cookie value and calls `session_by_token_hash`. No row, or an expired row, means anonymous. 3. For a valid session, `touch_session` updates `last_seen`. The user is loaded with `user_by_id`. 4. The page handler builds its view. Repo pages call `authorize(actor, repo, role, Action::Read)` first (section 6). 5. A POST form includes the session's `csrf_token` as a hidden field. The handler compares it in constant time. A missing or wrong token is rejected before any side effect. 6. On success, the handler performs the write through `Store` and redirects (PRG). 7. First run: while `user_count() == 0`, `/setup` creates the first admin. After that the route is closed. ### 5.2 Smart-HTTP fetch and push 1. Client sends `GET /{owner}/{repo}/info/refs?service=git-upload-pack` (or `git-receive-pack`). A trailing `.git` on the repo segment is stripped. 2. Auth: for upload-pack on a public repo, anonymous is allowed. Otherwise the request needs `Authorization: Basic`. The username is ignored. The password is a `forge_` token. The token is hashed and checked with `token_by_hash`. Failure gives 401 with `WWW-Authenticate: Basic`. 3. `authorize()` runs for Read (fetch) or Write (push). A private repo the caller cannot see gives 404. 4. The handler spawns `git --stateless-rpc --advertise-refs` through `service_command`, with `GIT_PROTOCOL` set from the `Git-Protocol` header. It writes the pkt-line header `# service=git-`, then `0000`, then streams git's stdout. Response has `Cache-Control: no-cache`. 5. Client sends `POST /{owner}/{repo}/git-upload-pack` or `/git-receive-pack`. If `Content-Encoding: gzip`, the body is inflated first. 6. Request body streams to git's stdin and git's stdout streams back. The route has no timeout and no body-size middleware. Nothing buffers a whole push. 7. For push, receive-pack runs the pre-receive and post-receive hooks (section 5.4). Objects land in quarantine until pre-receive accepts them. 8. Protocol v2 clients go straight to POST after the GET. ### 5.3 SSH fetch and push 1. Client connects to `ssh_listen` (default `0.0.0.0:2222`). russh runs the handshake under `auth_timeout` and `inactivity_timeout`. 2. `auth_publickey`: `GitBackend::authenticate_key(algorithm, key_blob)` does an exact byte match on `ssh_keys.key_blob`. `ssh-rsa` with SHA-1 is disabled. On success, `last_used_at` is touched. 3. Client sends an `env` request. Only `GIT_PROTOCOL` is accepted. Other names are ignored or refused. 4. Client sends `exec`. `parse_exec` accepts only `git-upload-pack` or `git-receive-pack` with a single-quoted path. Quotes and newlines inside the path are rejected. No shell is used. 5. `GitBackend::prepare(user_id, service, repo_path, git_protocol)` resolves the repo, runs `authorize()`, and builds the git `Command` through `forge-git`. On `Err(msg)`, the message goes to stderr and the exit status is 1. 6. The child's stdin and stdout are piped to the SSH channel. For receive-pack, `FORGE_*` env vars are set (section 5.4). 7. When git exits, the server sends `exit_status` and closes the channel. ### 5.4 Pre-receive and post-receive hooks over the unix socket Setup at boot: `write_hook_shims()` writes `/hooks/pre-receive` and `post-receive` (mode 0755) as shims that run `exec hook `. `core.hooksPath` in the managed gitconfig points there. Users cannot write hooks. Push: 1. The `forge` process spawns receive-pack with `FORGE_INTERNAL_SOCKET`, `FORGE_INTERNAL_TOKEN` (random per boot), `FORGE_ACTOR_ID`, `FORGE_REPO_ID`, and `FORGE_PROTOCOL` (`http` or `ssh`). 2. Git runs `pre-receive` with ` ` lines on stdin. Git has placed the new objects in quarantine (`GIT_QUARANTINE_PATH`, `GIT_OBJECT_DIRECTORY`, `GIT_ALTERNATE_OBJECT_DIRECTORIES`). They are invisible to other processes until the hook accepts. 3. The `hook pre-receive` client reads stdin and POSTs JSON to `/hook/pre-receive` on `/internal.sock`. Body: `{repo_id, actor_id, protocol, updates:[{old,new,ref}], quarantine_env:{...}}`. Header: `X-Forge-Internal-Token`. 4. The internal router checks the token in constant time. It then applies policy: - reject pushes to archived repos - reject deleting the default branch - reject force-pushes to the default branch, using `rev-list --max-count=1 ^` with the `quarantine_env` vars set on the command. Any output means a force push. - reject ref names with a leading `-` or containing `..` 5. The response is `{allow, message}`. The hook prints `message` to stderr and exits 0 if allowed, 1 if not. A non-zero exit rejects the whole push. 6. On accept, git moves the objects out of quarantine and updates refs. Git updates refs in one transaction. 7. Post-receive: `hook post-receive` POSTs the same updates to `/hook/post-receive`. The handler calls `record_push_event` for each ref. Replication (M5), webhooks (M5), PR ref updates (M4), and cache invalidation (M2) attach here. Post-receive cannot reject. 8. Hook failures are fail-closed for pre-receive. If the socket is unreachable, the push is rejected. ### 5.5 User-content domain for raw files (M2) Raw and user-uploaded files are served from `user_content_base_url` (for example `raw.`), a separate origin from the app. 1. Request arrives on the user-content host. The path is parsed as owner, repo, ref, and file path. 2. Repo lookup, then `authorize(actor, repo, role, Action::Read)`. Anonymous is allowed for public repos. A private repo the caller cannot see gives 404. 3. Every path component is checked for symlinks before reading, not only the last one (research §4). 4. The blob is read through `gix` and streamed. HTML and SVG are served with `Content-Disposition: attachment`. 5. Every response carries `X-Content-Type-Options: nosniff` and a strict CSP. 6. Cookies from the app origin are never sent to this origin. Session auth is not accepted on this host. --- ### 5.6 Response compression Topcoat's builder-wide compression is off on the public router, so it cannot compress git responses by mistake. `crates/forge-web/src/compression.rs` (`ResponseCompression`) runs instead. It gzips or brotlis a response only when all of these hold: - the content type is text (HTML, CSS, plain text, JSON, SVG, JavaScript); - the body size is known and between 32 bytes and 4 MiB (streamed bodies are never compressed); - there is no `Content-Encoding` or `Content-Range` header; - the request path is not a git smart-HTTP, LFS, raw, archive or download route (`/info/refs`, `/git-upload-pack`, `/git-receive-pack`, `/info/lfs`, and `raw`, `archive`, `download` segments). Content types outside the list (`application/x-git-*`, `application/octet-stream`, archives, images) are never compressed. The raw user-content listener has compression off because its ETag and Content-Length must match the stored bytes. ## 6. Authorization model One function: `forge_core::authz::authorize(actor: Option<&User>, repo: &Repo, collaborator_role: Option, action: Action) -> Result<(), Denied>`. - `Action`: `Read`, `Write`, `Admin`. - `Denied`: `NotFound` or `Forbidden`. - Callers: web pages, smart-HTTP, SSH `prepare`, the user-content domain, and hooks. Nothing else decides access. Rules (in order): 1. The owner or an instance admin has all actions. 2. A public repo allows `Read` to anyone, including anonymous. 3. A private repo with no role for the actor returns `NotFound`. 4. Role `Read` or `Triage` allows `Read`. 5. Role `Write` or `Maintain` allows `Read` and `Write`. 6. Role `Admin` allows all actions. 7. The actor can see the repo but lacks the right: `Forbidden`. 8. Archived repos deny `Write` with `Forbidden`. Status mapping: `NotFound` becomes 404 everywhere, including git and SSH. `Forbidden` becomes 403 on web and after authentication. For git over HTTP, an unauthenticated request that needs auth gets 401 with the Basic challenge. A private repo is never revealed by a 403. Any future go-get-style endpoints follow the same rule. Roles: `Read`, `Triage`, `Write`, `Maintain`, `Admin` (per repo, from `collaborators`), plus instance admin. Org teams come later (research §10). --- ## 7. Git layer ### 7.1 Command builder (`forge-git::GitCommand`) Rules: - No shell. Ever. `tokio::process::Command` only. - `env_clear()`, then `PATH=/usr/bin:/bin`, `HOME=`, `GIT_CONFIG_NOSYSTEM=1`, `GIT_CONFIG_GLOBAL=/gitconfig`. - `arg()` takes only `&'static str` flags, which are trusted. - `git_dir()` places `--git-dir=` before the subcommand. - `user_revs()` appends `--end-of-options`, then the revisions. It rejects any revision starting with `-`. - `paths()` appends `--` then the paths. - `env()` takes only `&'static str` keys. - User input reaches git only through `GitCommand`. - Repo-local config is not trusted (research §4). Submodules are never recursed. ### 7.2 Managed gitconfig (`/gitconfig`) Written at boot by `write_managed_gitconfig()`. Keys: - `core.hooksPath=/hooks` - `receive.fsckObjects=true`, `transfer.fsckObjects=true` - `core.fsync` and `core.fsyncMethod` set explicitly. The default `committed,-loose-object` risks losing recent work (research §16). - `uploadpack.allowFilter=true`, `uploadpack.allowRefInWant=true` - `receive.advertisePushOptions=true` - `protocol.file.allow=user` (research §4) - Later: `uploadpack.hideRefs` and `transfer.hideRefs` for internal refs (M4 `refs/pull/*` as needed) ### 7.3 Repo path layout `/repos///.git`, where `` is the lowercased ULID. Created with `git init --bare` through the builder. Renames are DB-only, because the path depends on the id, not the name. ### 7.4 Maintenance and caches (M2 and later) - `git maintenance` with incremental repack and loose-objects, or `git repack --geometric=2 --write-midx`. - Commit-graph with changed-path Bloom filters (makes `git log -- path` fast). - Bitmaps for clone speed. - A pool of long-running `git cat-file --batch-command` processes, one per repo, for hot reads (research §7). - "Last commit per file" cache keyed by `(commit, path)` for tree views (research §5). - Archive cache, with no promise of stable checksums across git versions. - Forks: full copies first. Alternates or object pools are deferred, and pools must never be pruned (research §5). - Ref updates use `git update-ref --stdin` for atomicity. ### 7.5 Minimum git version - Boot calls `require_min_version(env, (2,38,0))`. Boot fails if git is older. - 2.38 is required for `merge-tree --write-tree` (M4). - In practice, track the latest git release for security fixes. Keep it patched (CVE-2024-32002 and CVE-2025-48384 are submodule hook-execution bugs, research §4). --- ## 8. Data safety and reliability ### 8.1 Topology - **Primary** (writes, serves users). - **Standby** at a different provider. It receives git mirrors, DB snapshots, and bundles. - **Immutable backups** at a third location and provider. - Provider diversity covers provider, payment method, contact email, and jurisdiction. Research §11 notes that account termination can happen without warning. Lesson from SourceHut (2024): the standby was under 30s behind, but backups were up to 12h old with no off-site copy. Lesson from OVH Strasbourg (2021): backups at the same site burned with the primary. ### 8.2 Git replication (single writer) 1. After post-receive, a replication job takes a per-repo lock and runs `git push --mirror` to each replica over SSH. Failures are recorded and retried with backoff. Push mirrors run after each push (M5). 2. Hourly reconcile: compare a ref checksum (sha256 of sorted `git for-each-ref --format='%(objectname) %(refname)'`) between primary and replica. On mismatch, re-mirror and run `git fsck`. 3. Daily `git bundle create --all` per repo to off-provider storage, then `git bundle verify`. 4. Weekly `git fsck --full` on replicas from cron. `git maintenance` has no fsck task. 5. Never `rsync` or `cp` a live repo or a live DB. ### 8.3 Metadata DB (Turso) - Turso supports only `synchronous` OFF or FULL. Use WAL with FULL. - Litestream is not verified on Turso files, and Turso says mixed SQLite and Turso access across processes is unsupported. Spike before relying on it (open question 2). - Turso Sync and CDC are documented only for Turso Cloud. Do not depend on them for self-hosting. - Verified path: `VACUUM INTO` snapshots every 5-15 minutes. Ship each snapshot to the standby and to backups. Keep a JSON or F3 export as a second format (F3: friendly forge format, JSON plus attachments, research §9). - Integrity: `PRAGMA quick_check` daily. `integrity_check` plus `foreign_key_check` weekly. Alert on any result other than `ok`. ### 8.4 Integrity, backups, and drills - Backups follow 3-2-1-1-0: 3 copies, 2 media, 1 offsite, 1 immutable, 0 verify errors. - Immutable storage: Backblaze B2 Object Lock in compliance mode, Cloudflare R2 bucket locks, or Wasabi compliance mode. - The server holds no delete credentials. Either a separate machine pulls the backups, or the server pushes with append-only keys (rest-server `--append-only`, or BorgBase append-only). - Tools: restic (mature, `check --read-data-subset`), rustic (Rust, restic-compatible, newer), borg 1.4 (stable). - Keys: add a second restic key and store keys offline. Lost key means lost data. - Instance key `forge.key` (encrypts sealed values) is backed up separately and escrowed offline. Without it, sealed secrets are gone. - Restore drill monthly: automated restore to a fresh VPS. Time it. That time is the real RTO. - Dead-man switch: healthchecks.io pings with `/start` and `/fail` for each backup and integrity job. A missed ping alerts. - Filesystem: btrfs (or ZFS with at least 4 GB RAM) for checksums, scrub, and snapshots. ext4 or XFS with tested backups is acceptable on small VPSes. - App files: write temp file, fsync it, rename it, fsync the directory. - After any unclean shutdown, run `git fsck` (GitLab runbook, research §11). ### 8.5 Manual failover with fencing 1. Stop writes on the primary (block the HTTP and SSH listeners, or put the instance in read-only mode). 2. Fence the old primary: revoke its replication and backup credentials and make sure it cannot accept writes, so it cannot come back as a second writer. Do this before promotion. 3. Confirm the standby's last `VACUUM INTO` snapshot and last ref reconcile. Record the data-loss window. 4. Restore the DB snapshot and repos on the standby. Run `quick_check` and `fsck`. 5. Promote the standby. Switch DNS. DNS and health checks alone do not enforce a single writer, so fencing in step 2 is required. 6. Record the incident and the actual RTO. --- ## 9. Security checklist Do: - Put `--end-of-options` before user revisions and `--` before paths. Reject names and refs that start with `-`. - Route all git calls through `GitCommand`. - Check every path component for symlinks when rendering, archiving, or writing. - Use exact, case-sensitive key lookup (Gitea auth bypass, research §2). - Disable `ssh-rsa` with SHA-1. Set handshake and auth timeouts (Gogs DoS, research §2). - Accept only `GIT_PROTOCOL` from SSH env requests (Gogs CVE-2024-39930, research §2). - Store tokens and session tokens as sha256 hashes. Compare in constant time. Show tokens once. - Keep tokens out of URLs and logs. - Return 404 for both missing and forbidden private repos. - Normalize names with NFKC and case-fold. Keep a unique index on the folded form. Keep a reserved-names list. Reject names ending in `.git` or `.atom`. - Serve raw and user content from a separate origin with `nosniff` and a strict CSP. Use `Content-Disposition: attachment` for HTML and SVG. - Sanitize rendered markdown with comrak then ammonia. Allow only http, https, and mailto links. - Use linear-time regex only (Rust `regex` crate). - Cap request bodies on non-git routes, diff sizes, blame and log time, and archive sizes. Git routes are the exception, with streaming and no cap. - For SSRF (mirrors, webhooks, migrations): resolve DNS, pin the IP, block private and link-local ranges, re-check after every redirect, and block `file://`. - Run as a dedicated service user. Keep the git binary patched. - Set `receive.fsckObjects`. - Rate limit per user and per IP, with writes weighted heavier. Gate open registration behind email verification. Don't: - Don't use a shell, `sh -c`, or string-built commands. - Don't use dumb HTTP or `http.getanyfile`. - Don't recurse into submodules or trust repo-local config. - Don't let users write hooks or config. - Don't hard-delete users. Reassign their issues and comments to a ghost user (M3). - Don't let the browser use the app origin for raw content. - Don't bypass branch rules through a non-PR path (the Gitea gap: required checks only applied to PR merges, research §3). - Don't rely on Turso Sync or Litestream until spiked. --- ## 10. Milestones ### M1: clone and push work (this plan) - Scaffold the workspace (`cargo add`, latest versions), config template, `.gitignore`, systemd unit, SCSS build. - Local accounts with argon2id. First user becomes admin, via `admin create-user` or `/setup` while zero users exist. Registration is `closed` by default. - Sessions, CSRF on every POST, `__Host-` cookie. Access tokens (`forge_`) and SSH keys with exact blob match. - Repo create with the ID-based path. Visibility, collaborators, archive flag. - Smart HTTP (`info/refs`, upload-pack, receive-pack, gzip, streaming) and SSH (exec allow-list, `GIT_PROTOCOL`, piping, exit status). - Pre-receive policy (archived, default-branch delete and force-push, bad ref names) and post-receive `push_events`. Minimal UI: setup, login, dashboard, new repo, repo page, settings, admin users, `/healthz`. - Tests: unit tests in core and git; end-to-end `git_e2e.rs` over HTTPS and SSH, with negative cases (bad token, cross-user private 404, forced default-branch push rejected, unknown SSH command rejected, traversal rejected, POST without CSRF rejected). ### M2: repo browsing and raw domain (implemented) Built: - **forge-git** (`browse.rs`, gix, blocking, called through `spawn_blocking`): ref listing, revision resolution to a full OID, splitting `/`, tree and blob reads from the object database, commit reads, and path validation. Symlinks are shown as their target text and never resolved. - **forge-git** (`history.rs`, git CLI, OIDs only): paginated `log`, `commit_diff` against the first parent with rename detection and the limits in section 10 below, one-walk `last_commits`, and `archive_command`. - **forge-core** (`raw_token.rs`): `sign_raw` / `verify_raw` for private raw links (10 minute TTL). - **forge-web** `render/`: `markdown.rs` (comrak, then ammonia, relative links rewritten to the repo tree, blob, and raw bases) and `highlight.rs` (syntect, class-based output, served at `/assets/highlight.css`). - **forge-web** `raw.rs`: the user-content listener, bound by `boot.rs` on `user_content_listen`, with explicit routes only. - **forge-web** pages: `repo` (home and README), `tree`, `blob`, `refs` (branches and tags), `commits`, `commit`, `archive`. Routes on the app origin: - `/{owner}/{repo}`, `/{owner}/{repo}/tree/{*rest}`, `/{owner}/{repo}/blob/{*rest}` - `/{owner}/{repo}/commits/{*rest}` (`?page=`, 50 per page), `/{owner}/{repo}/commit/{oid}` - `/{owner}/{repo}/archive/{ref}.tar.gz` and `.zip` - `/{owner}/{repo}/branches`, `/{owner}/{repo}/tags` Routes on the user-content origin: `GET /{owner}/{repo}/{*rest}`, where `rest` is `/`. Limits: blob view 1 MiB; highlighting skipped above 512 KiB or 20,000 lines; raw files up to 8 MiB held in memory, larger ones streamed; diffs 300 files, 2,000 lines per file, 20,000 lines total; last-commit lookup is one bounded `git log` walk (2,000 commits, 3 second timeout), with no cache. Known gaps: - Footnote ids are not prefixed `user-content-` (heading ids are). - No Dockerfile grammar in syntax highlighting. - A background commit-graph and maintenance run after push is being added. It is not in place yet. - The app CSP header is computed once per process (`OnceLock`), so a change to `user_content_base_url` needs a restart. - Tree, blob, commits, and diff views through `gix`, with `git log` and `git diff` as fallback where needed. - README rendering with comrak, then ammonia. Relative links and images rewritten to repo paths. - Syntax highlighting (syntect, or tree-sitter with matching core versions). - Raw file serving on `user_content_base_url`, with the headers in section 5.5. - Symlink-safe path resolution, and `cat-file --batch-command` pool. - Caches: last commit per file, archives. Maintenance schedule and commit-graph. ### M3: issues - Issues with per-repo numbers, state, author, timestamps, and edit history. - Comments on issues (polymorphic `comments` table), with ghost-user reassignment on account deletion. - Labels and milestones, with per-repo scope. - Issue list and detail pages, filters, and permissions through `authorize()`. - Markdown bodies sanitized as in M2. ### M4: pull requests - PR head stored as `refs/pull/N/head` in the base repo, updated on every push to the head branch in post-receive. - Mergeability checked with `git merge-tree --write-tree`, recomputed when base or head moves. Optional `refs/pull/N/merge`. - Three-dot diffs from the merge base, rename detection (`-M`), binary detection, lazy-loaded large files. - Reviews (approve, request changes, comment) and review comments anchored to commit, path, line, side, and hunk. Comments are marked outdated, not deleted. - Merge strategies: merge commit (`commit-tree` from merge-tree output), squash, rebase, fast-forward only. - `range-diff` for "what changed since the last review." AGit (`refs/for/`) is considered later. ### M5: webhooks, push mirrors, replication, backups, export - Webhooks: HMAC-SHA256 signature over the raw body, delivery id header, 10s timeout, retry queue, delivery log. - Pull mirrors on an interval with a minimum; push mirrors with sealed credentials, run after each push with backoff. SSRF rules from section 9 apply. - Git replication per section 8.2: per-repo lock, `push --mirror`, hourly reconcile, daily bundles, weekly `fsck --full` on replicas. - DB: `VACUUM INTO` snapshots every 5-15 minutes, integrity checks, JSON or F3 export. - Backups per section 8.4: immutable tier, append-only push, restore drills, dead-man alerts. - Manual failover runbook per section 8.5, tested once end to end. ### M6: custom CI runner and commit-status API - Commit-status API (`commit_statuses`: commit oid, context, state, target URL, creator). Required checks enforced on push and PR merge, not only on PR merge (research §3). - Webhooks from M5 drive CI job dispatch. - Custom runner: a separate process with a narrow protocol, that reports status back through the API. Runner credentials are scoped tokens. - Consider a Gitea-compatible API subset later, which would let Woodpecker, Renovate, and `tea` work (research §9). Decide when the API surface is known. - Log retention and secret masking for job output. --- ## 11. Open questions 1. **Project name.** "forge" is a placeholder. Renaming is a find-and-replace, but pick it before public URLs and crate names are used. 2. **Does Litestream work with Turso files?** Unverified. Turso says mixed SQLite and Turso access across processes is unsupported. Run a spike before relying on it for replication. Fallback is the `VACUUM INTO` path in section 8.3. 3. **Hosting providers.** Primary, standby, and backup providers must differ in provider, payment method, contact email, and jurisdiction (section 8.1). Candidates are not chosen. Confirm after reviewing the provider risk notes in research §11 (some are unverified).