rs_maps/DESIGN.md
2026-08-03 09:42:13 +01:00

428 lines
23 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!-- DESIGN.md -->
# Self-Hosted Mapping Server — Design Document
*Revision 2. Incorporates design-review decisions on TLS/subpath deployment, rootless systemd,
upload limits, auth surface, and GPX collision handling.*
## 1. Purpose
A self-hosted replacement for Google Maps' "save places / plan routes" functionality, built as a
single Rust binary. It shows an OpenStreetMap-based map in the browser, lets multiple logged-in
users tag locations (privately or shared), shows live GPS position, and lets users view/import GPX
routes from a shared folder on the server. Route *creation* with road/trail snapping is an
explicitly deferred, later addition.
Scale is two users (myself and my wife). Several decisions below are deliberately sized to that
fact and are called out where they'd need revisiting at larger scale.
## 2. Non-goals / explicitly deferred
- **GPX route creation with road/trail snapping.** Out of scope for v1. Will be added later as a
separate backend integration with a self-hosted router (BRouter is the leading candidate — light
weight, single Java process, good for hiking/cycling profiles). The API/DB design below should not
need rework to accommodate this later; it will likely add a `POST /api/routes/calculate` endpoint
that proxies to BRouter and writes the result into the shared GPX folder.
*Note:* BRouter is not storage-free — it requires pre-generated `.rd5` segment files at roughly
100200 MB per 5°×5° tile, so a country-sized area is a few GB. Small compared to the rejected
Nominatim/tile options, but budget for it.
- **Live turn-by-turn navigation.** Out of scope. The live-location feature is "show my current
position on the map," not routing guidance.
- **User/role administration.** No admin UI, no roles, no user deactivation flow. The only
"admin" lever is the registration token in `.env`.
- **Self-hosted tiles/geocoding.** Explicitly rejected earlier due to storage cost (full planet
Nominatim ≈1TB, tiles ≈120GB+). Tiles and search go live to OSM's own tile servers and Nominatim
from the *browser*, not proxied through the backend.
- **Session revocation on password change.** Decided against. Changing a password affects
subsequent logins only; existing sessions on other devices remain valid. See §8.
- **GPX file metadata (uploader, upload date).** Decided against. The folder is shared between two
trusted users; provenance has no value here. Filesystem only, no table.
## 3. Architecture overview
Three moving parts:
1. **Rust binary** (built with `axum`), running as a **rootless systemd user unit**. Serves:
- The API (auth, account management, markers, GPX folder listing/serving/upload)
- The frontend static assets, embedded into the binary at compile time via `rust-embed` (no
separate web server / no separate static file deployment step)
2. **PostgreSQL**, running as a **rootless podman quadlet** container under the same user. Stores
users, sessions, and markers.
3. **A shared GPX folder** on the host filesystem, owned by the same user. Plain files, no database
involved. Shared across all users — anyone logged in can list, read, and upload into it.
An **existing nginx reverse proxy** terminates TLS and forwards to the binary on
`127.0.0.1:8080`. The app is served under a **subpath** (e.g. `/maps`), configured via `BASE_PATH`.
See §9 for the prefix-handling contract, which is the fiddliest part of the deployment.
The **browser** talks to three different things directly:
- OSM tile servers (map imagery) — direct, no backend involvement
- Nominatim (`nominatim.openstreetmap.org`) (POI/address search) — direct, no backend involvement
- The Rust backend, via nginx (auth, markers, GPX) — the only thing that's actually "yours"
```
┌────────────────────────────────────────────────────────┐
│ OSM infrastructure (external) │
│ Tile servers Nominatim │
└───────────▲──────────────────▲───────────────────────────┘
│ tile requests │ search requests
│ (direct) │ (direct, submit-on-enter)
┌─────┴──────────────────┴─────┐
│ Browser │
│ Leaflet map · live-location │
│ toggle · marker/tag UI · │
│ GPX list & device file picker │
└───────────────┬─────────────────┘
│ HTTPS
┌────────────────▼───────────────────┐
│ nginx (existing, TLS terminated) │
│ location /maps → 127.0.0.1:8080 │
│ prefix NOT stripped │
└────────────────┬───────────────────┘
│ HTTP, loopback only
┌────────────────▼───────────────────┐
│ Rust binary (axum, systemd --user) │
│ - auth (register/login/logout/ │
│ reset) + account management │
│ - markers API │
│ - GPX folder list/read/upload │
│ - serves embedded frontend assets │
└───────┬───────────────────┬──────────┘
│ │
┌──────────▼─────────┐ ┌─────▼──────────────┐
│ Postgres (rootless │ │ Shared GPX folder │
│ podman quadlet) │ │ (plain filesystem, │
│ 127.0.0.1:5432 │ │ no DB, shared) │
│ users, sessions, │ │ │
│ markers │ │ │
└───────────────────────┘ └──────────────────────┘
```
## 4. Tech stack
| Concern | Choice | Notes |
|---|---|---|
| Web framework | `axum` | async, tokio/hyper-based, mature ecosystem |
| DB access | `sqlx` (postgres) | features are `runtime-tokio` + `tls-rustls` (two separate features in 0.8, not one combined one). rustls keeps a musl static build painless |
| Sessions | `tower-sessions` + Postgres store | opaque server-side session IDs; store crate owns its own migration |
| Password hashing | `argon2` | current best practice |
| Rate limiting | `tower-governor` | applied to auth + account endpoints (§8) |
| Email (password reset) | `lettre` (SMTP transport) | stub/logging transport acceptable for the build session |
| Frontend asset embedding | `rust-embed` | note: reads from disk in debug builds unless `debug-embed` is enabled |
| Frontend | Plain Leaflet + vanilla JS | talks to OSM tiles/Nominatim directly, and to the API for everything else |
| Config | `.env` via `dotenvy` | see §7 |
| Migrations | `sqlx migrate` (SQL files) | run against Postgres on startup or via CLI |
Pin actual crate versions at build time (`cargo add`, let it resolve) rather than trusting anything
written here.
## 5. Database schema
```sql
-- Required if the Postgres image predates 13; gen_random_uuid() is core from 13 onward.
-- CREATE EXTENSION IF NOT EXISTS pgcrypto;
-- users
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
username TEXT NOT NULL,
email TEXT, -- nullable: recovery channel only, see §8
password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
reset_token_hash TEXT, -- SHA-256 of the emailed token, never the token itself
reset_token_expires TIMESTAMPTZ
);
-- Case-insensitive uniqueness; multiple NULL emails permitted.
CREATE UNIQUE INDEX users_username_key ON users (lower(username));
CREATE UNIQUE INDEX users_email_key ON users (lower(email)) WHERE email IS NOT NULL;
-- markers (the "tags/flags" overlay)
CREATE TABLE markers (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
description TEXT,
lat DOUBLE PRECISION NOT NULL,
lon DOUBLE PRECISION NOT NULL,
category TEXT, -- free text in DB; fixed picker + "other" in the UI
color TEXT, -- optional marker color/icon hint
is_shared BOOLEAN NOT NULL DEFAULT false,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX markers_owner_idx ON markers (owner_id);
CREATE INDEX markers_shared_idx ON markers (id) WHERE is_shared;
-- updated_at: DEFAULT now() fires on INSERT only, so it needs a trigger (or an explicit
-- assignment in every UPDATE statement — trigger is harder to forget).
CREATE FUNCTION set_updated_at() RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER markers_updated_at BEFORE UPDATE ON markers
FOR EACH ROW EXECUTE FUNCTION set_updated_at();
-- sessions table: created/managed by the tower-sessions Postgres store crate.
-- Do not hand-roll; let the crate's migration handle it.
```
GPX files: **no table**, by decision. The backend lists/reads/writes files in `GPX_DIR`.
## 6. API surface
All paths below are relative to `BASE_PATH` (§9).
**Auth**
- `POST /api/register``{ username, email?, password, reg_token }`. `reg_token` compared against
`REG_TOKEN` in **constant time**. Password hashed with argon2. Email optional.
- `POST /api/login``{ username_or_email, password }`. Sets a long-lived session cookie. When the
user isn't found, still verify against a dummy argon2 hash so both paths take comparable time.
- `POST /api/logout` — destroys the session server-side, clears the cookie.
- `GET /api/me` — current user info if authenticated, 401 otherwise. Frontend uses this to decide
between login page and map.
- `POST /api/password-reset/request``{ email }`. Generates a high-entropy token, stores its
SHA-256 hash plus expiry, emails the link. Always returns success regardless of whether the
address exists.
- `POST /api/password-reset/confirm``{ token, new_password }`. Hashes the supplied token,
looks it up, checks expiry, updates the password hash, clears the token fields.
**Account management** (require auth; both require the current password so a borrowed unlocked
device can't silently take over the account)
- `POST /api/me/password``{ current_password, new_password }`. Verify, rehash, update. Then call
`session.cycle_id()` to rotate the current session ID. No other sessions are touched (§8).
- `POST /api/me/email``{ current_password, new_email }`. Verify, update. `409` on unique
violation.
**Config**
- `GET /api/config` — returns `{ tile_url, base_path }` so the frontend doesn't hardcode them.
May be folded into `GET /api/me` if preferred.
**Markers** (all require auth)
- `GET /api/markers` — the current user's own markers plus all `is_shared = true` markers from any
user. Accepts an optional `?bbox=minlon,minlat,maxlon,maxlat` filter — worth building now, since
adding it later is a breaking change.
- `POST /api/markers` — create.
- `PUT /api/markers/:id` — update (only if `owner_id` matches).
- `DELETE /api/markers/:id` — delete (only if `owner_id` matches).
**GPX** (all require auth)
- `GET /api/gpx` — list files in `GPX_DIR` (name, size, modified time).
- `GET /api/gpx/:filename` — serve raw file content. See §10 for mandatory filename validation.
- `POST /api/gpx/upload` — multipart. Writes into `GPX_DIR` with collision avoidance (§10).
Returns the final stored filename, which may differ from the one submitted.
The frontend also handles **client-side-only GPX viewing** — a file opened from the user's own
device via `<input type="file">` is parsed and rendered without touching the API, unless the user
chooses to save it to the shared folder.
## 7. Configuration (`.env`)
```
DATABASE_URL=postgres://user:pass@127.0.0.1:5432/mapserver
REG_TOKEN=<shared secret required at registration time>
SESSION_SECRET=<random 32+ bytes> # VERIFY AT BUILD TIME — see note below
SESSION_DURATION_DAYS=30 # "remember me" is the default behaviour, no checkbox
GPX_DIR=/home/mapserver/data/gpx
BIND_ADDR=127.0.0.1:8080 # loopback only; nginx is the only client
BASE_PATH=/maps # subpath the app is served under
PUBLIC_URL=https://your-domain.example # scheme+host only; reset links are PUBLIC_URL + BASE_PATH
TILE_URL=https://tile.openstreetmap.org/{z}/{x}/{y}.png
UPLOAD_MAX_BYTES=26214400 # 25 MiB
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USERNAME=...
SMTP_PASSWORD=...
SMTP_FROM=mapserver@your-domain.example
```
Notes:
- **`SESSION_SECRET` may be vestigial.** `tower-sessions` backed by a Postgres store issues a random
opaque session ID and keeps all state server-side — there may be nothing to sign. Confirm at build
time whether the chosen store consumes a key; if it doesn't, delete this line rather than shipping
config that implies a security property that isn't there.
- **`TILE_URL`** is config rather than hardcoded so a switch to a paid provider or a different layer
(e.g. OpenTopoMap for hiking) is an edit and a restart, not a recompile. Use
`tile.openstreetmap.org` directly — the old `{s}.tile.` subdomain-sharded form is deprecated.
- `.env` lives at `~/.config/mapserver/.env`, mode `0600`.
## 8. Auth and account behaviour
- **Registration**: username + password + registration token required; email optional. Token checked
against `REG_TOKEN` in constant time.
- **Email is a recovery channel only.** It has no other function. A user who supplies a bad address,
or none at all, is only removing their own ability to self-serve a password reset — no other
capability depends on it. Accordingly there is no verification flow and the column is nullable.
- **Login**: sets a session cookie. Sessions live in Postgres so restarts don't log anyone out.
Expiry per `SESSION_DURATION_DAYS`.
- **Cookie attributes**: `Secure` (TLS is terminated at nginx), `HttpOnly`, `SameSite=Strict`, and
`Path` set to `BASE_PATH` so the cookie doesn't leak to other apps on the same domain.
- **CSRF**: `SameSite=Strict` is the primary defence, sufficient for a single-origin app. Also check
the `Origin` header on mutating requests as a cheap second layer.
- **Logout**: destroys the session server-side and clears the cookie.
- **Password reset**: request → email with a time-limited, single-use token link → confirm screen.
Only the token's hash is stored. Requires working SMTP; `lettre`'s stub transport logging the URL
to stdout is fine for the build session, so provider signup doesn't block starting.
- **Password change does not revoke sessions.** *Deliberate.* Changing a password affects subsequent
logins only; sessions already established on other devices stay valid. The current session's ID is
rotated via `cycle_id()`, which invalidates a leaked ID for the device performing the change but
touches nothing else. The consequence, accepted knowingly: if a device with a live session is
lost, changing the password will not lock it out — that requires clearing session rows directly
(`DELETE FROM tower_sessions;` via psql). At two users this is an acceptable manual recovery path;
it would not be at larger scale.
- **Rate limiting** via `tower-governor` on `/api/login`, `/api/register`,
`/api/password-reset/request`, `/api/me/password`, and `/api/me/email`. A single shared
registration secret with unlimited attempts is guessable given enough time.
## 9. Deployment
### Subpath handling (the part most likely to go wrong)
The contract is: **nginx does not strip the prefix.** Paths are then identical on both sides of the
proxy and there is nothing to keep in sync.
```nginx
location /maps {
proxy_pass http://127.0.0.1:8080; # NO trailing slash, no rewrite
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /maps/api/gpx/upload {
client_max_body_size 25m; # nginx default is 1m and would reject first
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
On the axum side: `Router::new().nest(&base_path, app)`.
Four things must carry the prefix:
1. **Session cookie `Path`** = `BASE_PATH`.
2. **Frontend asset URLs** — inject `<base href="/maps/">` into `index.html` at serve time. Every
relative URL in the JS (`fetch("api/markers")`) then resolves correctly with no other changes.
Two gotchas: no leading slashes anywhere in the frontend, and the trailing slash on the `href` is
mandatory.
3. **Reset links** — derive as `PUBLIC_URL + BASE_PATH + "/reset?token=…"` rather than storing a
second full URL that can drift out of sync.
4. **SPA fallback** — must live *inside* the nest, so `/maps/reset?token=…` serves `index.html`
rather than 404ing.
### Upload size limits
Two limits sit in series and both must be raised, on the upload route only:
- **nginx**: `client_max_body_size 25m;` (default 1m).
- **axum**: `DefaultBodyLimit` defaults to **2 MB** and applies to the `Multipart` extractor,
producing a `413` before the handler runs. Apply
`.layer(DefaultBodyLimit::max(UPLOAD_MAX_BYTES))` to the upload route specifically — a 25 MB body
allowance on `/api/login` would be free memory exhaustion.
- In the handler, **stream** the field to disk (`while let Some(chunk) = field.chunk().await?`)
rather than `field.bytes().await`, which buffers the entire file in RAM.
25 MB is sized for a long GPS track with per-point elevation and timestamps (~20k points lands in
the 515 MB range).
### Rootless systemd
Both the binary and Postgres run as the same non-root user, as **user units** — which means
ordering between them works normally within the single user manager.
- Quadlet: `~/.config/containers/systemd/postgres.container`
- App unit: `~/.config/systemd/user/mapserver.service`
```ini
[Unit]
Description=mapserver
After=network-online.target postgres.service
[Service]
EnvironmentFile=%h/.config/mapserver/.env
ExecStart=%h/.local/bin/mapserver
Restart=on-failure
[Install]
WantedBy=default.target
```
- `WantedBy=default.target`, **not** `multi-user.target` (user units).
- **`loginctl enable-linger <user>`** — without it, user units don't start at boot and are killed on
logout. (Already in place on this server; noted for completeness.)
- Postgres quadlet: **named volume** for `PGDATA`, not a host bind mount — rootless podman maps
container UIDs into the subuid range and bind mounts hit permission errors that named volumes
avoid. Publish as `127.0.0.1:5432:5432`.
- `After=` does **not** guarantee Postgres is accepting connections. The binary must retry its
initial pool connection with backoff rather than exiting.
- `GPX_DIR` under a directory the same user owns outright.
## 10. GPX file handling
**Filename validation is mandatory** on both read and write, since filenames are user-supplied and
appear in a path parameter. Without it, `GET /api/gpx/../../etc/passwd` reads anything the service
user can read.
- Reject any name containing a path separator, or not matching `^[A-Za-z0-9 ._-]{1,120}$`.
- Join to `GPX_DIR`, then canonicalize and assert the result is still under `GPX_DIR`.
- Do not attempt to sanitize by string-replacing `..`.
**Collision avoidance** uses `create_new(true)`, which is atomic — it fails if the path exists, so
there's no check-then-write race:
```rust
fn reserve(dir: &Path, stem: &str, ext: &str) -> io::Result<(File, String)> {
for n in 0..1000 {
let name = if n == 0 { format!("{stem}.{ext}") } else { format!("{stem}-{n}.{ext}") };
match OpenOptions::new().write(true).create_new(true).open(dir.join(&name)) {
Ok(f) => return Ok((f, name)),
Err(e) if e.kind() == ErrorKind::AlreadyExists => continue,
Err(e) => return Err(e),
}
}
Err(io::Error::other("too many collisions"))
}
```
`track.gpx` becomes `track-1.gpx`, then `track-2.gpx`. The final name is returned in the upload
response so the UI can show what was actually saved. On a failed or aborted upload, delete the
reserved file rather than leaving a zero-byte stub.
## 11. Frontend feature list (v1)
- Login / register / forgot-password / reset-password pages
- Account page: change password, change email (both requiring current password)
- Map view: Leaflet, tile layer from `TILE_URL`, Nominatim-backed search box
- **Search must be submit-on-enter**, not search-as-you-type. Nominatim's usage policy prohibits
autocomplete-style querying. Send an identifiable `User-Agent`/`Referer`.
- Live location toggle: `watchPosition()` (not one-shot), on/off, shows a dot at current position
while enabled. Requires a secure context — works because nginx terminates TLS.
- Marker/tag management: click map (or use current location) to drop a marker, set
name/category/color/description, toggle "shared with others". Own markers plus all shared markers
are shown. Category is a fixed picker plus a free-text "other", stored as free text.
- GPX panel:
- List files from the shared server folder (`GET /api/gpx`), select one to render on the map
- "Open from this device" — local file picker, parses and renders client-side immediately
- Optional "save to server" on a device-opened file, uploading into the shared folder. Surface the
returned filename if it was renamed for collision avoidance.
## 12. Open items for the build session
Design-level review is complete; everything in §§311 is settled. What remains are things that can
only be resolved against a compiler:
- Pin actual crate versions (`cargo add`).
- Confirm whether the chosen `tower-sessions` Postgres store consumes `SESSION_SECRET`; delete it
from `.env` if not (§7).
- Confirm `sqlx` feature-flag names resolve as expected for the pinned version.
- Decide whether `rust-embed`'s `debug-embed` feature is wanted, so debug and release builds behave
identically.
- Obtain an SMTP provider, or ship with the stub transport and wire it later.
- Verify the Postgres image version supports `gen_random_uuid()` natively; add the `pgcrypto`
extension to the first migration if not.