I'm tired, and can't make new tricky stuff, but I can benchmark.

I want to know how fast the current code is at encoding and decoding leaf nodes. I want at least 10 MiB/s in each direction.

The benchmark:

  • create a leaf node with 19 byte keys and 32 byte values
  • node size is 64 KiB; that's 1285 pairs
  • how fast is an encode?
  • how fast is a decode of the encoded node?

Node is actually 68114 bytes.

First result:

  • encoding: 3.62 ms or
  • decoding: 18.6 ms

To convert to MiB/s:

68114 B / 3.62 ms = 1000 * 68114 B / 3.62 s = 17.9 MiB/s
68114 B / 18.6 ms = 3.49 MiB/s

Decoding needs optimizing.

  • Construct a large string, then call struct.pack/unpack: 20.7 us encoding, 17.5 us decoding. Much better. That's 3.1 GiB/s encode, 3.7 GiB/s decoding.

That's way fast enough. No need to optimize further.

Did similar changes for index nodes. Same kinds of speeds.