Reduce number of small buffer allocations for index builds

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Major - P3
    • 4.7.0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • Execution Team 2020-04-06, Execution Team 2020-04-20
    • 0
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      Index builds allocate a single, small buffer for each key generated. For index builds with small keys, much of the index build CPU time is spent just in memory allocation.

      It is generally faster to allocate fewer, larger memory regions than numerous, smaller ones. We can speed up index builds by allocating a few large buffers for key generation and sorting the pointers to keys within those buffers.

      This has the added benefit of having better cache locality for sorting since all keys will be nearby in memory.

            Assignee:
            Henrik Edin
            Reporter:
            Louis Williams
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: