Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-4163

Imported large CSV file into Compass creates document count mismatch

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 1.21.0
    • Affects Version/s: 1.20.5
    • Component/s: Compass, Import/Export
    • None
    • Environment:
      MacOS 10.14.6
      Compass 1.20.5 Stable
    • Iteration Yak

      Used Compass > Import Collection > CSV > local file of a very large CSV file (3.7GB) from http://prod.publicdata.landregistry.gov.uk.s3-website-eu-west-1.amazonaws.com/pp-complete.csv from 
      https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads

      creates error after 20 sec of attempting to complete import. After clicking on "Import" button to continue, it seems to try again. Screenshot of database collections shows "0" documents, but inspecting the collection shows that 2.9m documents were saved.

      Perhaps also give guidance to Compass users what the max size of an import file that could be attempted.

       

        1. Screen Shot 2020-02-24 at 2.03.46 PM.png
          Screen Shot 2020-02-24 at 2.03.46 PM.png
          83 kB
        2. Screen Shot 2020-02-24 at 2.05.46 PM.png
          Screen Shot 2020-02-24 at 2.05.46 PM.png
          321 kB
        3. Screen Shot 2020-02-24 at 2.17.08 PM.png
          Screen Shot 2020-02-24 at 2.17.08 PM.png
          905 kB
        4. Screen Shot 2020-02-24 at 2.17.25 PM.png
          Screen Shot 2020-02-24 at 2.17.25 PM.png
          1.02 MB
        5. Screen Shot 2020-02-24 at 2.40.14 PM.png
          Screen Shot 2020-02-24 at 2.40.14 PM.png
          469 kB
        6. Screenshot 2020-03-09 13.30.57.png
          Screenshot 2020-03-09 13.30.57.png
          489 kB
        7. Screenshot 2020-03-09 13.31.19.png
          Screenshot 2020-03-09 13.31.19.png
          189 kB
        8. Screenshot 2020-03-09 14.07.34.png
          Screenshot 2020-03-09 14.07.34.png
          202 kB

            Assignee:
            lucas.hrabovsky Lucas Hrabovsky (Inactive)
            Reporter:
            felicia.hsieh@mongodb.com Felicia Hsieh
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: