[pull] canary from vercel:canary - #1402
Merged
Merged
Conversation
#97705) ### What? Rearranges the bytes inside a key block so that the bytes a lookup's binary search compares are contiguous, instead of interleaved with key and value payload the search never reads. | block kind | before | after | | --- | --- | --- | | fixed, `HashThenKey` | `[hash\|key\|val]*n` | `[hash]*n` then `[key\|val]*n` | | fixed, `KeyOnly` | `[key\|val]*n` | `[key]*n` then `[val]*n` | | variable, `HashThenKey` | `[type\|pos]*4B` then `[hash\|key\|val]…` | `[hash\|type\|pos]*12B` then `[key\|val]…` | ### Why? The binary search routines are a random access pattern through the hash/key ranges, we only need to look at value payloads when we find a plausible match. So by organizing all the 'random access data' in denser memory range we optimize cache/page fault behavior. `get` perf impacts are small but real. | shape | in-block search before → after | as % of a ~1070 ns `get()` | | --- | --- | --- | | key=8, value=4 inline | 166 → 146 ns | 1.9% | | key=4, small value ref | 144 → 119 ns | 2.4% | The reason the impacts are small is mostly because the 'working sets' of the benchmarks are not large enough and the page size of macos is quite large.
### What? Three small, localized optimizations to the read path. No behaviour or format changes. **Avoid contended atomic refcounts on the mmap** - before we would inc/dec the mmap for the index block, the key bloc and then optionally the inline value of the key block - Now we front-load the index block 'parsing', borrow the key block (when it doesn't need decompression), and return inline values from key blocks by _value_ instead of reference **Hoist block cache access out of loops.** - This is most impactful on misses, where we were touching the OnceLocks for every MetaFile **Make scanned data dense.** - Reorganize `MetaEntry` to separate the hash ranges from the rest of the data - This improves the cache effectiveness of our hot 'entry scanning' loops since the stride is now 16 bytes instead of 144. ### Performance Interleaved A/B of this branch against its base (`LARGE_DB=1`, same machine, alternating runs, quiet host). Criterion medians: | benchmark | base | this branch | delta | rounds won | | --- | ---: | ---: | ---: | --- | | `get` miss, 20 commits uncompacted | 600 ns | 568 ns | **−5.4%** | 7/8 | | `get` hit, 20 commits uncompacted | 1.415 µs | 1.398 µs | −1.2% | 4/4 | | `get` hit, compacted | 996 ns | 979 ns | −1.7% | 2/4 | | `get` miss, compacted | 125.0 ns | 124.5 ns | −0.4% | 2/4 | Note: as single threaded benchmarks we are not really detecting any benefits from the refcount work, under concurrent reads we should expect an impact
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )