perf(pg-protocol): encode length-prefixed strings in a single pass#3681
Open
kylecannon wants to merge 2 commits into
Open
perf(pg-protocol): encode length-prefixed strings in a single pass#3681kylecannon wants to merge 2 commits into
kylecannon wants to merge 2 commits into
Conversation
Add Writer.addInt32PrefixedString, which writes a value's Int32 byte-length prefix immediately followed by its UTF-8 bytes, computing Buffer.byteLength once. The previous `addInt32(Buffer.byteLength(s)).addString(s)` pairing scanned each string three times. Used for Bind parameter values and the SASL initial response. Wire output is byte-identical; addInt32/addString are unchanged for other callers. Benchmark (packages/pg-protocol/bench/write-bench.js, alternating-sampled vs base): bind(2 small) +16% bind(10 mixed) +23% bind(unicode) +14% full insert seq +9% Adds a multi-byte unicode bind unit test asserting the Int32 prefix equals the UTF-8 byte length, not the char length. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
charmander
reviewed
Jun 3, 2026
Collaborator
|
Other than that, looks good. Can you share the benchmark you used? |
Co-authored-by: Charmander <~@charmander.me>
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
Writer.addInt32PrefixedString, which writes a value's Int32 byte-length prefix immediately followed by its UTF-8 bytes, computingBuffer.byteLengthonce. The previousaddInt32(Buffer.byteLength(s)).addString(s)pairing scanned each string three times (once for the prefix length, again insideaddString, then the encode). The new method is used in two places:serializer.writeValues), the common write path for parameterized queries.Why
Bind is on the hot path for every parameterized query, and the cost of the redundant
Buffer.byteLengthscans grows with parameter size. Doing the length prefix and the UTF-8 write in one pass removes two of the three string scans for each string parameter.Correctness
The serialized wire output is byte-identical to the previous
addInt32(Buffer.byteLength(s)).addString(s)sequence:Buffer.byteLength), not the character count, exactly as before.this.ensure(4 + len)is called before capturing the local buffer reference, so a backing-buffer growth cannot write through a stale reference.addInt32andaddStringare left unchanged for all other callers.No public API or behavior changes. The new method is purely additive.
Benchmark
Measured with an in-process serialization microbenchmark (no network), alternating between this branch and base each round to cancel thermal drift:
The gain grows with parameter size, since the eliminated scans are proportional to string length.
Testing
The
pg-protocoltest suite passes. This change adds a multi-byte unicode bind unit test that asserts the Int32 length prefix equals the UTF-8 byte length rather than the character count (the exact property the single-pass write must preserve, which the existing ASCII-only bind tests could not catch).