# Durable streams

Append-only JSON logs addressed by URL, hosted by Electric SQL. Not Cloudflare Durable Objects — they share a word and nothing else (a durable stream is byte storage; a Durable Object is an isolate with state). This page is protocol substrate; arbe's envelope, client boundary, and event contract live in [streams](./streams.md).

```
PUT  /stream/{path}                     # create (JSON mode by default)
POST /stream/{path}                     # append (returns Stream-Next-Offset header)
GET  /stream/{path}?offset=X            # read range (JSON array)
GET  /stream/{path}?offset=X&live=long-poll | live=sse   # tail
```

Offsets are opaque, monotonic, lexicographically sortable strings — store them, never parse or construct. Sentinels: `-1` (full replay), `now` (tail). Historical reads are immutable byte ranges, CDN-cacheable. Live SSE cycles ~60s for CDN connection collapsing — clients reconnect from the last `control` event's `streamNextOffset`.

```
arbe-thread-{threadId}     # arbe's only stream type today — one per thread (threadStreamId() in @arbe/core/schemas/thread.ts)
```

Each item is an `ArbeStreamEntry<ArbeThreadPayload>`; that envelope and payload contract live in [streams](./streams.md). `@arbe/streams/client` provides the two transport clients (raw service + scoped proxy); `@arbe/core/entries` is the thread-aware consumer (`ensureThreadStream`, `postThreadEntry`, `readThreadEntries`) used from CLI, www, and dispatch. The web chat UI reads and writes via `@arbe/core/client`, decoded by `apps/www/src/lib/chat-stream.ts` and rendered by `Chat.svelte`.

Worth knowing:

- **Idempotent producers** — `Producer-Id` / `Producer-Epoch` / `Producer-Seq` headers. Epoch fences zombies with `403`; a retry of an accepted `(id, epoch, seq)` returns a dedup'd success. `IdempotentProducer` in the TS client handles batching and pipelining.
- **Forking** — `PUT` with `Stream-Forked-From: <path>` branches a new independent stream at `Stream-Fork-Offset` (defaults to source tail) without copying history. Deleting the source soft-deletes it until the last fork is gone. Fits conversation branching, deterministic replay, and producer handoff.
- **Closure** — `Stream-Closed: true` on a final POST seals a stream. Once closed it stays closed; reads still work.
- **TTL** — `Stream-TTL` (relative seconds) or `Stream-Expires-At` (RFC 3339), mutually exclusive at create.
- **Retention** — servers may drop old data. On `410 Gone`, reset to `-1` or `now`.
- **Live modes** — `?live=sse` for JSON/text streams, `?live=long-poll` for binary or simple request-response.

Delete is hard. `deleteThread` (called by `arbe thread delete` and `DELETE /api/threads/:id`) drops the durable stream first, then the row — so a stream-side failure leaves the row in place for retry rather than orphaning a stream. `deleteHouse` snapshots child thread ids, lets the row delete cascade (FK on house), then sweeps each thread's stream best-effort — a stream-side hiccup is logged, never strands the row.

The wider durable-streams ecosystem (Durable State, StreamDB, StreamFS, Yjs provider, TanStack AI / Vercel AI SDK transports, Durable Proxy, multi-language clients) lives upstream. arbe uses only the protocol + `@durable-streams/client` + the TanStack AI transport for chat. Backend today: Electric Cloud ([electric-sql.cloud](https://dashboard.electric-sql.cloud)).

`@arbe/streams` also ships a self-host backend as an alternative to Electric Cloud: `gateway.ts` is a single-token auth proxy (`DS_SECRET`, file-backed storage, `make deploy` to a sprite) in front of the embedded `server.ts` stream server, which also backs the package's tests and `demo/` scripts.

Code: `@arbe/streams/client`, `packages/streams/{gateway,server}.ts`, `@arbe/core/entries`, `packages/core/schemas/stream-events/thread.ts`, `packages/core/schemas/thread.ts` (`threadStreamId`). Upstream docs: [durable-streams.com](https://durable-streams.com), [protocol spec](https://github.com/durable-streams/durable-streams/blob/main/PROTOCOL.md).<br>
See [streams](./streams.md), [threads](./threads.md), [system/dispatch](./dispatch.md).