Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-7850

Add more accuracy tests for complex queries and aggregations

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: None
    • None
    • Developer Tools
    • 3
    • Not Needed
    • Iteration Zephyrosaurus

      Let's add 5 (or more) tests to our generative ai accuracy tests. Let's aim to make these more complex generations and hopefully hit some of the negative feedback received from https://docs.google.com/spreadsheets/d/1vhzZorEFSLsP5UYJzI6Jc0Tv-Q5sWvE_ny9E0npig8k/edit#gid=1482967694 
      Accuracy tests: https://github.com/mongodb-js/compass/blob/main/packages/compass-generative-ai/scripts/ai-accuracy-tests.ts 
      Ideally these will serve as a way for future model evaluations to show their improvements.

       

      Let's be sure we do one with nested documents and nested arrays.

            Assignee:
            rhys.howell@mongodb.com Rhys Howell
            Reporter:
            rhys.howell@mongodb.com Rhys Howell
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: