--- title: "Designing a Local-First Trust Boundary for AI Agents" description: "Foxl keeps durable state, tools, and credentials local while selected model context crosses the provider boundary you configure." author: "Foxl Team" date_published: "2026-03-18" date_modified: "2026-07-16" canonical_url: "https://foxl.ai/blog/why-local-first" markdown_url: "https://foxl.ai/blog/why-local-first/index.md" image: "https://foxl.ai/blog/local-first-cover.png" social_image: "https://foxl.ai/blog/social/why-local-first.png" content_type: "Deep dive" topics: ["Local-first architecture","Trust boundaries","Data privacy"] products: ["Foxl Agent","Foxl Relay"] --- # Designing a Local-First Trust Boundary for AI Agents > Local-first is a boundary, not a claim that no byte leaves the machine. Foxl keeps tools, files, credentials, and durable conversation state local while sending only the selected model context through the provider path you configure. - Author: Foxl Team - Published: 2026-03-18 - Last reviewed: 2026-07-16 - Reading time: 11 minutes - Canonical HTML: [https://foxl.ai/blog/why-local-first]() ![A trust-boundary diagram showing durable conversation state, tools, and credentials staying on the local machine while only the selected model context crosses to the configured provider.]() "Local-first" is often shortened to "nothing leaves the machine." That is not an honest description of an agent using a cloud model. A model cannot reason about a prompt it never receives, a browser cannot open a website without contacting it, and an integration cannot read mail without calling the mail service. The useful architectural claim is narrower: where does durable state live, where do tools execute, where are credentials resolved, and which selected data crosses each trust boundary? Foxl keeps the desktop agent's durable state and execution authority on the user's machine. It sends model context through the provider path the user configured, and it treats remote control as a separate encrypted transport. ![A trust boundary places durable files, credential stores, conversation state, and tools on the local machine; selected context crosses to the configured model provider, while a separate end-to-end encrypted tunnel connects a remote client]() ## Local-first describes three different properties ### Durable state is local In the desktop app, conversations, messages, tool records, schedules, agent snapshots, provider configuration, and permission rules are persisted in SQLite. In the default production layout, the database is `~/.foxl/data/pilot.sqlite`. Human-readable memory and workspace documents live under `~/.foxl/workspace/`, and the browser profile used by desktop automation is also stored on the local machine. `FOXL_HOME` can move that root. Those files are the desktop agent's durable sources of truth, not a cache of a server-side conversation. The [conversation-store incident report]() shows why Foxl makes that authority explicit. Foxl's cloud services still keep the data needed to operate an account, including identity, billing and usage records, paired-device metadata, connection state, and tunnel audit events. Local-first does not mean that the product has no server-side data. It means the server is not the durable home of the desktop workspace, tool history, or conversation database. ### Tool execution is local The embedded agent server listens on loopback. File reads and writes use the desktop filesystem, shell and terminal tools spawn processes under the current OS user, and browser automation drives the [dedicated Foxl browser profile]() on that machine. A hosted agent worker does not need a copy of the user's home directory in order to run those tools. Local execution is not the same as offline execution. A shell command can call a network service. The browser sends requests to every site it opens. Slack, Outlook, and other integrations call their respective APIs. The local runtime decides and performs the action, but each external destination remains its own trust boundary. ### Credential custody is local until use Provider keys entered in the desktop app are encrypted in the local database. Other providers can use environment variables, the AWS credential chain, vendor authentication files, or an OS credential store. The desktop resolves those credentials and uses them to authenticate the outbound request. On a direct desktop route, the provider key is not sent to Foxl's model relay, while a Foxl account token is necessarily presented to Foxl services for account-backed models and remote access. Credentials are transport authority, not model context. They are not normally inserted into the prompt. That boundary can still be defeated by a permitted tool that reads a secret file, prints an environment variable, or returns sensitive page content. If that output enters the conversation, it can be included in a later model request. Local custody reduces ambient exposure; it does not make credential leakage impossible. ## The model boundary is deliberate egress Before a model invocation, the desktop constructs a context from a bounded set of sources. Depending on the conversation and provider, that can include system instructions, enabled workspace context, relevant conversation history, the current message and attachments, tool definitions, and results from tools already run. The model does not receive the whole disk merely because the agent can access it. "Selected context" does not mean that the user confirms every byte separately. Selection happens through the context builder, workspace configuration, conversation history, attachments, and tool activity. A file that is never read and is not part of configured workspace context is not uploaded by default. Once file content or a browser result is brought into a model-visible tool result, that content may cross the provider boundary on the next turn. ### The configured provider determines who can read it Foxl's [provider configuration guide]() identifies which credentials and route each option uses. At the data boundary, the choices reduce to these paths: - **Local model:** with a local provider such as Ollama, context stays on the machine, although it still crosses a local process boundary. - **Direct cloud provider:** with desktop BYOK, OAuth, or the user's AWS account, the request travels from the desktop to that provider. Foxl's model relay is not in that path. The provider receives the context in plaintext after transport security terminates and handles it under the provider's own retention and abuse-monitoring policies. - **Foxl account model:** when the selected model uses Foxl credits, the desktop sends the model request to the Foxl relay. The relay authenticates the account, resolves the model, enforces entitlement and usage policy, and forwards the request to the upstream model service. It can process the request and response while proxying them. It is not a durable conversation store, but "not stored as a conversation" is different from "not visible in transit." This distinction is the core correction to the claim that all data stays on the laptop. Durable state can remain local while selected context leaves for inference. Users who require the model context to remain local must choose a local model and avoid tools or integrations that contact external services. ## The remote tunnel is a separate path Remote access adds a transport boundary before the model boundary. The path is: ``` remote browser -> encrypted tunnel -> desktop runtime -> configured model path ``` The browser and desktop perform an ephemeral ECDH exchange on P-256, derive a shared key with [HKDF](), and create [AES-256-GCM envelopes]() for application payloads. The [remote relay]() forwards those envelopes through an account-scoped tunnel. It does not receive either endpoint's private key or the derived session key. ### What the tunnel relay can see End-to-end encryption hides payload content, not the control plane. The relay authenticates the account and routes a connection, so it can see account and device authentication material, device name and platform, browser fingerprint and User-Agent, online and last-seen state, connection timing, public key-exchange messages, routing and request identifiers, envelope types, ciphertext length, and traffic timing. Some tunnel audit events also record an IP address. It can reject, delay, or drop a connection even when it cannot decrypt a protected payload. ### What it cannot see from an encrypted payload For data carried inside the E2E envelope, the tunnel relay cannot derive the plaintext conversation, file content, tool input, or tool result from the ciphertext. AES-GCM also authenticates the encrypted bytes, so modification inside that envelope is detected by the receiving endpoint. The browser and desktop do see plaintext, which makes both endpoints part of the trust boundary. The guarantee is envelope-scoped. Control fields, metadata, or a compatibility path sent outside the E2E envelope are relay-readable and rely on hop-by-hop transport security. If the endpoints have not established a shared key, no E2E confidentiality claim applies to that payload. The key-exchange claim assumes that each endpoint receives the other endpoint's authentic ephemeral public key. Foxl derives a short shared-key fingerprint for verification. Comparing that fingerprint is the check against active key substitution; without an independent comparison, encryption alone should not be treated as protection from a malicious control plane. The E2E tunnel also does not extend into model inference. If a remote request reaches the desktop and the configured model is a Foxl account-backed model, the desktop decrypts the tunnel payload and then creates a separate model request through the Foxl model relay. The tunnel path cannot read the first leg; the model-proxy path can process the second. They may use the same service domain, but they are different trust relationships. ## The threat model, stated plainly Local-first removes a broad class of ambient cloud custody. A breach of the relay's account database should not produce the user's desktop SQLite database, workspace, browser profile, or provider credential store. A passive tunnel intermediary should not receive the plaintext of an E2E-protected remote session. A direct model provider does not need shell access to the machine merely because it receives a prompt. It does not protect against every relevant attacker: - Malware or another process running with the user's privileges can read local files, observe the browser, or call the local agent. - A compromised desktop or remote browser sees content before encryption or after decryption. - The configured cloud model provider receives selected context, and a Foxl account model also places the Foxl model relay in that data path. - Prompt injection can influence tool requests. A permission grant does not prove that the requested action is wise. - Local conversation and workspace data are not generally encrypted by Foxl as a whole. Full-disk encryption, OS account security, updates, and physical security remain the user's responsibility. ## Permissions are policy gates, not a sandbox Foxl evaluates tool calls against deny, allow, and ask rules. Shell, process, terminal, and selected Git mutations default to approval, while several read-only operations can run automatically. Built-in rules deny some destructive command patterns, and user or session rules can allow a matching operation without asking again. This is materially different from saying that every tool call requires confirmation. Unmatched tools can be allowed, persistent rules broaden authority, and approved commands run with the desktop user's real OS permissions. The gate can stop a call before execution. It cannot roll back an email, a pushed commit, a changed file, or a command that partially completed. ## Failure behavior follows data placement - **Model or provider unavailable:** local durable state remains readable, but a model-dependent turn cannot complete. A local model is required for inference without a cloud provider. - **Desktop offline:** the remote client loses access to local tools and the desktop conversation store. When the Foxl relay is otherwise available, the web app can fall back to cloud chat with browser-local history. That fallback is a different execution mode, not a remote copy of the desktop agent. - **Relay unavailable:** remote access and Foxl account-backed model routing are unavailable. The local database and workspace continue to exist, and direct or local provider paths do not depend on the tunnel transport. - **Machine asleep or powered off:** local tools and schedules cannot execute until the machine can run again. A remote relay cannot execute them on the desktop's behalf. - **Failure after a tool call:** a later model error or dropped connection does not undo an earlier external side effect. Retrying may require checking whether the first attempt already succeeded. ## Backups become part of the architecture When the primary copy is local, the owner also owns recovery. The remote relay is not a backup of the desktop conversation database or workspace. A practical backup plan covers `~/.foxl/data/pilot.sqlite`, `~/.foxl/workspace/`, and any custom skills or other local assets that matter. Conversation export is useful for portable records, but it is not a complete application backup. Take backups while Foxl is stopped or with a tool that can snapshot an open SQLite database coherently, then test restoration. Protect the backup as sensitive data. Provider keys stored by the desktop are encrypted with machine-derived material, so a restore to another machine may require re-authentication or re-entering credentials. Browser sessions deserve the same caution and should not be copied to an untrusted backup destination. ## The tradeoff is control for responsibility Local-first narrows centralized data custody, keeps tool authority near the resources it acts on, makes durable memory inspectable, and lets the user choose a local, direct, or relayed model path. In exchange, the endpoint must be secured, powered, maintained, and backed up. Remote access depends on an authenticated relay, and cloud inference still depends on a provider that receives the context required to answer. The accurate contract is therefore not "no data leaves your laptop." It is: desktop state, tool execution, and local provider credential stores stay under the user's machine boundary; selected model context leaves through the configured provider path; and protected remote-control payloads cross a separate end-to-end encrypted tunnel whose relay still observes control metadata. That is a boundary a user can reason about, audit, and choose, rather than a privacy slogan that fails on the first model request. ## References and further reading 1. [Foxl security model]() - Documentation 2. [Model providers and data paths]() - Documentation 3. [Desktop relay setup]() - Documentation