I'm tired, and can't make new tricky stuff, but I can benchmark.
I want to know how fast the current code is at encoding and decoding leaf nodes. I want at least 10 MiB/s in each direction.
The benchmark:
- create a leaf node with 19 byte keys and 32 byte values
- node size is 64 KiB; that's 1285 pairs
- how fast is an encode?
- how fast is a decode of the encoded node?
Node is actually 68114 bytes.
First result:
- encoding: 3.62 ms or
- decoding: 18.6 ms
To convert to MiB/s:
68114 B / 3.62 ms = 1000 * 68114 B / 3.62 s = 17.9 MiB/s
68114 B / 18.6 ms = 3.49 MiB/s
Decoding needs optimizing.
- Construct a large string, then call struct.pack/unpack: 20.7 us encoding, 17.5 us decoding. Much better. That's 3.1 GiB/s encode, 3.7 GiB/s decoding.
That's way fast enough. No need to optimize further.
Did similar changes for index nodes. Same kinds of speeds.