We are running 3 Mongo Instances with Replica sets enabled on Windows 2008 R2 codebase environment.
Mongo has only 1 database with approximate 3 GB in size.
Run, MongoDump.EXE on primary instance machine to take backup of the database.
MongoDump.EXE command-line:
mongodump.exe –host 127.0.0.1:27017 –out <Path_To_Som_Directory_On_Local_Machine> -vvvvv
Scenario:
1. All 3 instances are running fine
2. Run mongodump.exe on PRIMARY instance machine with above specified command-line
3. MongoDump.EXE starts the dumping process
4. MongoDump.EXE shows log entry “doing snapshot query” fir currently being processed collection (“doCollection” method under dump.cpp)
5. MongoDump.EXE shows progress in percentage for currently being dumped collection
6. While mongodump.exe is taking dump and showing percentage progress – mongod.exe goes down on primary instance (we can use db.shutdownServer() to simulate this)
7. Now, mongodump.exe is stuck in infinite loop – it keeps trying to receive data from Network Socket – as mongod.exe is down – it keeps throwing exception back
8. Mognodump.exe now never exits, unless we manually kill it.
9. MongoDump.EXE shows following output while being stuck in infinite loop
a. SocketException: remote: 127.0.0.1:27017 error: 9001 socket exception [0] server [127.0.0.1:27017]
b. Socket recv() conn closed? 127.0.0.1:27017
Expected behaviour:
- MongoDump.EXE should exit gracefully if the mongo instance to which it is connected goes down.
Actual:
MongoDump.EXE never exits and gets stuck in infinite loop while trying to receive data from Network Socket
I have created dump (.dmp) of MongoDump.EXE while stuck in this process.
You can access dump file at: https://dl.dropbox.com/u/43222617/mongodump_1544_2012-10-08_18-40-31-135_061c.dmp