Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-4163

Imported large CSV file into Compass creates document count mismatch

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 1.21.0
    • Affects Version/s: 1.20.5
    • Component/s: Compass, Import/Export
    • None
    • Environment:
      MacOS 10.14.6
      Compass 1.20.5 Stable
    • Iteration Yak

      Used Compass > Import Collection > CSV > local file of a very large CSV file (3.7GB) from http://prod.publicdata.landregistry.gov.uk.s3-website-eu-west-1.amazonaws.com/pp-complete.csv from 
      https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads

      creates error after 20 sec of attempting to complete import. After clicking on "Import" button to continue, it seems to try again. Screenshot of database collections shows "0" documents, but inspecting the collection shows that 2.9m documents were saved.

      Perhaps also give guidance to Compass users what the max size of an import file that could be attempted.

       

        1. Screen Shot 2020-02-24 at 2.03.46 PM.png
          83 kB
          Felicia Hsieh
        2. Screen Shot 2020-02-24 at 2.05.46 PM.png
          321 kB
          Felicia Hsieh
        3. Screen Shot 2020-02-24 at 2.17.08 PM.png
          905 kB
          Felicia Hsieh
        4. Screen Shot 2020-02-24 at 2.17.25 PM.png
          1.02 MB
          Felicia Hsieh
        5. Screen Shot 2020-02-24 at 2.40.14 PM.png
          469 kB
          Felicia Hsieh
        6. Screenshot 2020-03-09 13.30.57.png
          489 kB
          Lucas Hrabovsky
        7. Screenshot 2020-03-09 13.31.19.png
          189 kB
          Lucas Hrabovsky
        8. Screenshot 2020-03-09 14.07.34.png
          202 kB
          Lucas Hrabovsky

            Assignee:
            lucas.hrabovsky Lucas Hrabovsky (Inactive)
            Reporter:
            felicia.hsieh@mongodb.com Felicia Hsieh
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: