Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-8369

Increase the character size restriction for query and aggregation genai

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.44.6
    • Affects Version/s: None
    • Component/s: None
    • None
    • Developer Tools
    • 3
    • Not Needed
    • Iteration Milky Way

      Currently if folks use the generative ai feature in Compass with a collection with documents with a lot of fields, we'll show an error:
      "Sorry, your collections have too many fields to process. Please try 
      using this feature on a collection with smaller documents."
      PROMPT_TOO_LONG error from mms
      This is because we don't want to exceed the token limit of open ai. We recently changed from gpt-3.5-turbo to gpt-4o. Lets increase the character restriction now to a higher amount that matches what might be tokenized then: https://platform.openai.com/tokenizer 
      Context window tokens here: https://platform.openai.com/docs/models/gpt-4o 
      4o: 128,000 tokens
      3.5-turbo: 16,385 tokens

       

      Since we want the final validation to be on the server lets do 10x what the server limit is at. The server does some further changes to schema and sample documents so we should let it be the decider. The frontend should only prevent uploading massive schemas and documents in the request.

       

      Let's also update the messaging for the PROMPT_TOO_LONG error. We can make it more transparent what the issue is, and remove the plural 's' on collections.

            Assignee:
            ruchitha.rajaghatta@mongodb.com Ruchitha Rajaghatta
            Reporter:
            rhys.howell@mongodb.com Rhys Howell
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: