FTS indexes pass c-strings to tokenizer so they ignore data in strings past the first nul byte

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Won't Fix
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Index Maintenance, Querying
    • None
    • ALL
    • Query Execution 2021-03-08, Query Execution 2021-03-22, Query Execution 2021-04-05, Query Execution 2021-04-19, Query Execution 2021-05-03, Query Execution 2021-05-17, Query Execution 2021-06-14
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      "Luckily" the same bug exists in both indexing and querying. Unfortunately, because it is in indexing, it will require a version bump to fix and we would need to keep the old code around.

       

      This also adds a relatively slight overhead because it is computing strlen rather than just using the size stored in the BSONElement. But that cost is trivial relative to the actual tokenization, so I don't think it is a good motivation alone to fix this.

            Assignee:
            Mickey Winters
            Reporter:
            Mathias Stearn
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: