Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2666

Mongo Restore Continuously Failing

    • Type: Icon: Bug Bug
    • Resolution: Community Answered
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      Hi Mongo DB Tools Support Team, 

      We recently upgraded from a single node MongoDB cluster running version 3.4.x to a multi-node cluster running 3.6.x. As part of the upgrade, we are in need of running on-demand DB refreshes from one environment to another and are using the mongodump and mongorestore utilities with the arguments mentioned below.  

      Some important notes:

      • The commands run inside a Docker container running Mongo version 3.6.1.   
      • The variables are passed using environment variables at runtime.  
      • The backup/restore size all of the collections about 15GB. 

       

      # Backup 
      mongodump \ 
        --host "${SOURCE_DB_HOST}" \ 
        --port "${DB_PORT}" \ 
        --username "${SOURCE_DB_UN}" \ 
        --password "${SOURCE_DB_PW}" \ 
        --db "${DB_NAME}" \ 
        --authenticationDatabase "${DB_NAME}" \ 
        --out "${OUTPUT_DIR}" \ 
        --quiet
      
      # Restore 
      mongorestore \ 
        --drop "${OUTPUT_DIR}/${DB_NAME}" \ 
        --host "${DESTINATION_DB_HOST}" \ 
        --port "${DB_PORT}" \ 
        --username "${DESTINATION_DB_UN}" \ 
        --password "${DESTINATION_DB_PW}" \ 
        --db "${DB_NAME}" \ 
        --authenticationDatabase "${DB_NAME}"

       

      The backup runs successfully but the restore fails every time on a larger collection (hence I've marked this a P2).

      As a debugging step, we've tried the solutions in the following articles to no avail: 

      https://stackoverflow.com/questions/34142673/mongo3-0-restore-fails-with-error-failed-restore-error-insertion-error-eof 

      https://jira.mongodb.org/browse/TOOLS-939

      https://medium.com/synapseinc/tricks-to-restore-large-mongo-collections-on-small-machine-bc5c27c9076b

      https://groups.google.com/g/mongodb-user/c/Ms2MXCajreg/m/OsW6ixcSCgAJ?pli=1

      What can we try next to resolve this issue? Let me know if you need more information to guide us in the right direction and I'll do my best to provide it for you. 

            Assignee:
            tim.fogarty@mongodb.com Tim Fogarty
            Reporter:
            patht004@apps.disney.com Tarpan Pathak
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: