--- title: "The Checksum Byte That Ate Your Long Answers" description: "A checksum byte broke Foxl's text-based stream parser. This incident report covers binary framing, bounded parsing, and fixture tests." author: "Foxl Team" date_published: "2026-06-26" date_modified: "2026-07-16" canonical_url: "https://foxl.ai/blog/stream-cutoff-binary-frames" markdown_url: "https://foxl.ai/blog/stream-cutoff-binary-frames/index.md" image: "https://foxl.ai/blog/stream-cutoff-binary-frames.png" social_image: "https://foxl.ai/blog/social/stream-cutoff-binary-frames.png" content_type: "Incident" topics: ["Binary protocols","Streaming","Amazon Bedrock"] products: ["Foxl Relay"] --- # The Checksum Byte That Ate Your Long Answers > Foxl replies were truncating mid-sentence on long answers. The cause was a single byte. Our relay decoded Amazon Bedrock's binary event-stream as if it were text and brace-scanned for JSON - so a CRC32 checksum byte equal to 0x22 (a double-quote) flipped the parser into string mode and silently swallowed the rest of the stream. A captured 32 KB stream dropped 70 of 119 events. Here is the binary-framing parser that fixed it, and the fixture test that keeps it fixed. - Author: Foxl Team - Published: 2026-06-26 - Last reviewed: 2026-07-16 - Reading time: 5 minutes - Canonical HTML: [https://foxl.ai/blog/stream-cutoff-binary-frames]() ![A stream of Anthropic SSE events with deltas dropped after a CRC byte equal to 0x22 flipped the text parser into string mode.]() ## Incident summary The relay truncated some Bedrock responses because it decoded an AWS event-stream body as UTF-8 and searched it for JSON objects. The payloads were JSON, but the enclosing protocol was binary. Length fields and checksums could therefore change the text scanner's state and prevent later events from being emitted. The replacement parser consumes declared binary frame lengths from a rolling byte buffer. A captured 32,254-byte Bedrock stream is the regression fixture. It contains 119 messages, all of which are recovered with no leftover bytes under the tested chunk boundaries. ## Impact Once the text scanner lost framing alignment, later content deltas and terminal events were not forwarded. The client received a valid prefix but not the complete response or its final `message_stop`. A related boundary defect could discard a partial JSON object between reads, corrupting streamed tool input before the binary parser was introduced. ## The protocol on the wire Bedrock's [`InvokeModelWithResponseStream` API]() uses `vnd.amazon.eventstream`. The [Amazon event-stream encoding specification]() defines each message as a 12-byte prelude, optional headers, a payload, and a trailing checksum: ``` [4B total length][4B headers length][4B prelude CRC] [headers][payload][4B message CRC] ``` On the Claude path, the payload is a JSON wrapper whose `bytes` field decodes to an Anthropic event. On the [ConverseStream path](), the payload is the Converse event body and its event name is stored in the `:event-type` header. Both paths use the same outer binary framing. ## Root cause The old parser tracked braces, quotes, and string state over the entire decoded response. That model assumed bytes outside a payload were harmless text. They were not. A framing byte equal to `0x22` looked like a quote, and `0x7B` looked like an opening brace. Either value could make real payload boundaries disappear inside the scanner's state. The captured fixture documents 956 quote-valued bytes and 121 opening-brace-valued bytes in its framing. The incident record shows the text scanner dropping 70 of the 119 messages. Retaining a partial text object between reads, as covered by `INC-014`, fixed one boundary loss but could not make binary framing safe to scan as text. ## Resolution: rolling binary parsing `extractEventStreamMessages` receives a `Uint8Array` containing the previous unconsumed tail followed by the next network read. It parses only complete messages: 1. Keep the entire buffer when fewer than 12 prelude bytes are available. 2. Read the total and header lengths as big-endian unsigned integers. 3. Leave a declared frame untouched when its final byte has not arrived yet. 4. Slice headers and payload by offset, then advance by the declared total length. 5. Return every complete message and the unconsumed tail for the next read. ``` const { messages, rest } = extractEventStreamMessages(buffer); buffer = rest; ``` The payload-only wrapper serves the Claude InvokeModel path. The header-aware result serves ConverseStream models by decoding `:event-type` while skipping other AWS header value types at their declared widths. ## Frame bounds and CRC limitation The parser accepts a declared total length from 16 through 8,000,000 bytes. It stops without consuming data when the value is smaller, larger than that 8 MB bound, or when the declared header length exceeds the total length. It also stops when the complete frame is not yet buffered. Because a binary stream cannot be safely resynchronized from an arbitrary byte, the parser does not guess a new starting offset. The current parser does not validate either CRC. It skips the four-byte prelude CRC and excludes the four-byte message CRC from the payload, but it never computes or compares their values. `INC-021` makes this explicit by constructing frames with both CRC fields set to zero and expecting them to parse. The 8 MB bound is therefore a framing sanity check, not an integrity check. A frame with plausible lengths and an invalid CRC is not rejected today. ## Test evidence - `INC-017` asserts that the decoded fixture is exactly 32,254 bytes and yields exactly 119 messages. - Chunk sizes 16, 64, 256, 1,024, 4,096, and 32,254 bytes each recover all 119 messages with zero leftover bytes. - One-byte reads force every frame across many boundaries and still recover all 119 messages with zero leftover bytes. - The recovered sequence contains one `message_start`, one `content_block_start`, 114 `content_block_delta` events, one `content_block_stop`, one `message_delta`, and one `message_stop`. - An incomplete eight-byte prelude is returned intact as the tail. A 64-byte corrupt prelude with an out-of-range length stops without throwing or consuming bytes. - `INC-014` retains split JSON objects, including tool input divided across repeated 37-character reads. `INC-021` recovers all seven header-aware ConverseStream events at chunk sizes 1, 4, 16, 64, 256, and the complete stream length. ## Current guarantee The covered guarantee is boundary-safe parsing for valid-shaped frames whose declared lengths are within the configured bound. The regression suite covers the captured Claude stream, pathological single-byte reads, and header-aware ConverseStream frames. CRC integrity remains outside that guarantee until checksum validation is added. ## References and further reading 1. [Amazon event stream encoding specification]() - Reference 2. [Choose and configure models in Foxl]() - Documentation