Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-62500

Improve "ks" VM function to support all BSON types

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • QE 2022-01-24

      1) Check out the master branch at or after git rev 50db8e9573e1 ("SERVER-62434 Implement query optimizer based on Path algebra and Cascades").

      2) Build.

      3) Launch mongod with the new optimizer enabled:
      ./build/debug/install/bin/mongod --setParameter 'featureFlagCommonQueryFramework=true'

      4) Launch mongo shell and run the following commands:

      > db.adminCommand({setParameter:1, internalQueryEnableCascadesOptimizer: true});> let t = db.foo;
      > t.drop();
      > t.createIndex({'a': 1});
      > t.createIndex({'b': 1});
      > for (let i = 0; i < 10; i++) { t.insert({a: null, b: null, c: i}); }
      > for (let i = 0; i < 100; i++) { t.insert({a: i + 10, b: i + 10, c: i + 10}); }
      > t.aggregate([{$match: {'a': null, 'b': null}}])

      You should get the following error message: 

      Error: command failed: .. "PlanExecutor error during aggregation :: caused by :: unsuppored key string type"

      This error is happening because the "ks" VM builtin function does not support all types yet.

      The implementation of the "ks" VM builtin function is provided by the ByteCode::builtinNewKeyString() method in "src/mongo/db/exec/sbe/vm/vm.cpp".

      The goals of this task are: (1) to improve the implementation of the "ks" VM builtin function so that it supports all types; (2) to write unittests for the "ks" VM builtin function passing in every possible BSON type; and (3) to add some jstest coverage for queries that pass different types to the "ks" VM builtin function.

      For goal #3, I'd recommend taking a look at some the existing jstests in the "jstests/aggregation/abt" folder, in particular:

      python3 buildscripts/resmoke.py run --suites=cqf jstests/aggregation/cqf/index_intersect.js
      python3 buildscripts/resmoke.py run --suites=cqf jstests/aggregation/cqf/index_intersect1.js
      python3 buildscripts/resmoke.py run --suites=cqf jstests/aggregation/cqf/nonselective_index.js
      python3 buildscripts/resmoke.py run --suites=cqf jstests/aggregation/cqf/partial_index.js
      python3 buildscripts/resmoke.py run --suites=cqf jstests/aggregation/cqf/selective_index.js
      

      Note that the "ks" VM builtin function does not need to support SBE types that do not have a corresponding BSON type. This is means that the "ks" VM function does not need to support TypeTags::LocalLambda, TypeTags::ksValue, TypeTags::pcreRegex, TypeTags::timeZoneDB, TypeTags::jsFunction, TypeTags::shardFilterer, TypeTags::collator, TypeTags::ftsMatcher, or TypeTags::sortSpec (if any of these types are passed in, the "ks" VM function should either return Nothing or it should throw an error).

      Likewise, when two or more SBE types represent the same BSON type, such as TypeTags::Object and TypeTags::bsonObject, such types should be encoded in the same way (i.e. given an Object and and a bsonObject that contain the same values, their encodings should be indistinguishable from each other). 

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            andrew.paroski@mongodb.com Drew Paroski
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: