Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-11919

Add Tokenizer to JSON parser

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: Internal Code

      The JSON parser currently reads directly from the JSON string buffer, converting to BSON as it goes using recursive descent. This ticket is about adding a Tokenizer as an initial stage before doing the conversion to BSON. This may improve performance in some cases, since we could potentially reduce the number of string comparisons needed to determine which "$<keyword>" we are using, or which "<constructor>()" we are using for extended JSON types. This also may facilitate better error messages.

      The fact that the JSON parser doesn't Tokenize as a separate stage also means that we cannot use parseNumberFromString, which is currently our standard way to read numbers out of strings, but requires the length of the number to be known beforehand.

            Assignee:
            backlog-server-platform DO NOT USE - Backlog - Platform Team
            Reporter:
            sverch Shaun Verch
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: