Support 64 (or more) indexes

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Done
    • Priority: Major - P3
    • 1.5.2
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      The current code supports up to 40 indexes per collection. I think that we can support 64 without too much extra work, and much much more if you're willing to use some of the reserved bytes in Namespace. AFAICT, the 40 index restriction is twofold:

      1. multiKeyIndexBits is a bitwise storage of the multi-key state for each index, and it uses an unsigned long long, limiting to 64 indexes

      2. the first 10 indexes are stored in the Namespace, the next 30 are stored in an extra block, currently limiting to 40 total indexes, however forethought was given to this, and there exists the capability to chain additional extra blocks.

      The attached patch implements the extra block chaining. It accounts for the possibility of 10 more "extra" blocks by naming them $extra, $extr0, $extr1, $extr2 ... $extr9. You could actually get another 300 using this, for a hard limit of 340. However, because of the multiKeyIndexBits limitation, we're really limited to just 64 indexes. I implemented the full chaining for future expansion.

      If you're interested in 128 indexes, just take 8 or more bytes from the reserved array, and create a multiKeyIndexMoreBits[] and carry the logic into the multiKey functions. I did not do this because (a) the extra 24 indexes should be enough for our needs, (b) those reserved bytes are precious few and once you use them, they're gone, (c) I know we're crazy to want 64 indexes, and I think whoever wants 128 is even crazier.

      This patch is not yet complete. It dies in db/pdfile.h +284 on an assertion "ofs >= headerSize()". But I've only spent 2.5 hours on this so far, and it's late, and I wanted to get my work in progress in front of others to review and comment.

        1. mongo-64-indexes.patch
          8 kB
          Doug Green
        2. mongo-64-indexes.patch
          8 kB
          Doug Green
        3. mongo-64-indexes.patch
          7 kB
          Doug Green
        4. mongo-64-indexes-backport.patch
          13 kB
          Doug Green

            Assignee:
            Dwight Merriman
            Reporter:
            Doug Green
            Votes:
            2 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: