Designing Connection Ownership for Always-On AI Agents

Incident summary

Foxl v0.4.7 replaced User-Agent-based session labels with a stableclient_id for each installation and browser profile. It also introduced connection fencing for the desktop tunnel. The identity change fixed duplicate account sessions, but the tunnel work exposed follow-on regressions in v0.4.8 and v0.4.9.

The underlying mistake was treating device identity, connection ownership, and request ownership as one problem. They are separate. A stable client identity answers which installation signed in. A connection generation answers which socket may publish new state. A request id answers which socket may finish work that was already dispatched.

Impact

Before v0.4.7, the Account page derived session labels from User-Agent strings. The same installation could accumulate rows after logins or client changes, while two machines running the same browser version were difficult to distinguish. Revocation therefore operated on a session list that did not reliably represent devices.

The tunnel regressions had different symptoms. In v0.4.8, a stalled desktop loopback call could leave the relay waiting for its 10-second request timeout while status still showed the desktop as connected. In v0.4.9, reconnects could accumulate web-client rows, abort requests already in flight, and recycle a healthy status socket during token refresh or app resume.

Release sequence

v0.4.7: stable identity and initial fencing

Each installation and browser profile began sending a stable client_id on login and refresh. Existing duplicate rows could collapse to that identity, and revocation could target one installation. Concurrent refreshes from two app surfaces also became idempotent within a short concurrency window instead of being mistaken for refresh-token reuse.

The tunnel assigned each desktop WebSocket connection a connection_id. Offline writes were conditional on that id, so a delayed close from an older socket could not clear the replacement socket's online state.

v0.4.8: bounded local calls

When the desktop exhausted local network ports, its call to the local Foxl server could stall without producing either a response or an error. v0.4.8 bounded that call and returned an explicit failure. It also corrected response matching so a desktop reply could still find its pending relay request after a reconnect.

v0.4.9: reconnect ownership

The next release closed three remaining reconnect paths. The relay now preserves and forwards the web client fingerprint, then replaces the prior socket with the same fingerprint. A new desktop generation no longer fails every pending request merely because it connected. The web app also keeps a healthy status socket open across token refresh and resume, a lifecycle boundary discussed in the desktop agent field guide.

The important correction was narrower than removing fencing. Superseded sockets must not publish unsolicited state, but a superseded socket may still carry the valid response to a request that was sent before the reconnect.

The current three-part contract

1. Liveness. An HTTP heartbeat updates recency, but it does not mark the tunnel online. If D1 says a device is online, /tunnel/status confirms that state against the Durable Object. New desktop clients send a WebSocket heartbeat every 45 seconds; Cloudflare records the automatic response timestamp, and the production freshness window is 115 seconds. Older clients without that timestamp fall back to the socket's open state.
2. Generation ownership. One desktop connection id is active. A new generation fences older sockets from unsolicited messages and status changes. A delayed close from an inactive generation is a no-op. For a generation-tagged socket, D1 can clear online state only when the stored connection id matches. If the active generation closes, the hub may promote a healthy fenced fallback.
3. In-flight request correlation. HTTP requests, chat streams, and API streams are tracked by requestId. A response with a still-pending id is accepted from the socket that received the work even if that socket has since been fenced. An unmatched message from an inactive socket is discarded.

if (!correlatesToPending && !isActiveDesktop(ws)) return;

Regression evidence

INC-020 verifies status truth. A paired device with no desktop WebSocket remains offline after three HTTP heartbeats, and /tunnel/status agrees with/tunnel/api: status is offline and the API returns503 OFFLINE. The same suite verifies that a live desktop socket reports online, pushes online and offline transitions, and becomes offline when its WebSocket heartbeat timestamp goes stale.

INC-023 verifies generation and correlation behavior. It closes the active generation and confirms that a healthy fenced socket is promoted. It verifies that a remote disconnect closes both active and fallback sockets, repeated web connects with one fingerprint do not accumulate client sockets, and an API response from the original desktop still completes after a second desktop connects.

Result

One session per device is an identity invariant, not a complete tunnel design. The current implementation keeps that invariant separate from live-socket truth, active-generation ownership, and response correlation. Reconnects can replace ownership without rewriting device identity or discarding work that the previous generation already accepted.