1) Setup a config server, a mongos and 3 replicaset members.
2) Run the primary node in a shell and in foreground (without --fork).
3) Run attached rstest.sh to create an insert load via mongos to the primary.
4) Stop the primary node with Ctrl-Z.
5) Wait forever
6) Notice that rstest.sh is stuck and not inserting anything anymore, even if a new primary is available and known to mongos.
7) press Ctrl-C in the rstest.sh shell. (twice)
hingo@hingo-sputnik:~/hacking/rstests$ ./rstest.sh
Mon Jun 17 10:43:18 EEST 2013
MongoDB shell version: 2.4.4
connecting to: hingo-sputnik:27000/rstest
bye
Mon Jun 17 10:43:19 EEST 2013
*****************************************************
Mon Jun 17 10:43:20 EEST 2013
MongoDB shell version: 2.4.4
connecting to: hingo-sputnik:27000/rstest
--> Notice many minutes pass by:
^CMon Jun 17 10:46:03.942 Assertion: 13111:field not found, expected type 2
0x7495d1 0x71189b 0x711ddc 0x5f3c76 0x5dc554 0x7f514e48d4a0 0x7f514f23f14a 0x738aa3 0x738ab9 0x73cd05 0x73566c 0x7375db 0x737a94 0x64ce0f 0x67ab37 0x63dd4a 0x657b11 0x705efc 0x6e8a70 0x82ef62
mongo(_ZN5mongo15printStackTraceERSo+0x21) [0x7495d1]
mongo(_ZN5mongo11msgassertedEiPKc+0x9b) [0x71189b]
mongo() [0x711ddc]
mongo(_ZNK5mongo11shell_utils18ConnectionRegistry30killOperationsOnAllConnectionsEb+0x1526) [0x5f3c76]
mongo(_Z10quitNicelyi+0x24) [0x5dc554]
/lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7f514e48d4a0]
/lib/x86_64-linux-gnu/libpthread.so.0(recv+0x6a) [0x7f514f23f14a]
mongo(_ZN5mongo6Socket5_recvEPci+0x13) [0x738aa3]
mongo(_ZN5mongo6Socket11unsafe_recvEPci+0x9) [0x738ab9]
mongo(_ZN5mongo6Socket4recvEPci+0x75) [0x73cd05]
mongo(_ZN5mongo13MessagingPort4recvERNS_7MessageE+0x8c) [0x73566c]
mongo(ZN5mongo13MessagingPort4recvERKNS_7MessageERS1+0x1b) [0x7375db]
mongo(ZN5mongo13MessagingPort4callERNS_7MessageES2+0x34) [0x737a94]
mongo(_ZN5mongo18DBClientConnection4callERNS_7MessageES2_bPSs+0x4f) [0x64ce0f]
mongo(_ZN5mongo14DBClientCursor4initEv+0xb7) [0x67ab37]
mongo(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xea) [0x63dd4a]
mongo(_ZN5mongo18DBClientConnection5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xa1) [0x657b11]
mongo(_ZN5mongo9mongoFindEPNS_7V8ScopeERKN2v89ArgumentsE+0x42c) [0x705efc]
mongo(_ZN5mongo7V8Scope10v8CallbackERKN2v89ArgumentsE+0xb0) [0x6e8a70]
mongo() [0x82ef62]
Mon Jun 17 10:46:03.958 JavaScript execution failed: Error: field not found, expected type 2 at src/mongo/shell/query.js:L78
^CMon Jun 17 10:46:14.853 Assertion: 13111:field not found, expected type 2
0x7495d1 0x71189b 0x711ddc 0x5f3c76 0x5dc554 0x7f514e48d4a0 0x7f514f23bd82 0x6ff469 0x792b79 0x7f514f237e9a 0x7f514e54accd
mongo(_ZN5mongo15printStackTraceERSo+0x21) [0x7495d1]
mongo(_ZN5mongo11msgassertedEiPKc+0x9b) [0x71189b]
mongo() [0x711ddc]
mongo(_ZNK5mongo11shell_utils18ConnectionRegistry30killOperationsOnAllConnectionsEb+0x1526) [0x5f3c76]
mongo(_Z10quitNicelyi+0x24) [0x5dc554]
/lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7f514e48d4a0]
/lib/x86_64-linux-gnu/libpthread.so.0(pthread_cond_wait+0xc2) [0x7f514f23bd82]
mongo(_ZN5mongo15DeadlineMonitorINS_7V8ScopeEE21deadlineMonitorThreadEv+0x3b9) [0x6ff469]
mongo() [0x792b79]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7f514f237e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f514e54accd]
Mon Jun 17 10:46:14.858 terminate() called in shell, printing stack:
0x7495d1 0x5dc36e 0x7f514eddd846 0x7f514eddd873 0x7f514eddd96e 0x711941 0x711ddc 0x5f3c76 0x5dc554 0x7f514e48d4a0 0x7f514f23bd82 0x6ff469 0x792b79 0x7f514f237e9a 0x7f514e54accd
mongo(_ZN5mongo15printStackTraceERSo+0x21) [0x7495d1]
mongo(_Z11myterminatev+0x3e) [0x5dc36e]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f514eddd846]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f514eddd873]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f514eddd96e]
mongo(_ZN5mongo11msgassertedEiPKc+0x141) [0x711941]
mongo() [0x711ddc]
mongo(_ZNK5mongo11shell_utils18ConnectionRegistry30killOperationsOnAllConnectionsEb+0x1526) [0x5f3c76]
mongo(_Z10quitNicelyi+0x24) [0x5dc554]
/lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7f514e48d4a0]
/lib/x86_64-linux-gnu/libpthread.so.0(pthread_cond_wait+0xc2) [0x7f514f23bd82]
mongo(_ZN5mongo15DeadlineMonitorINS_7V8ScopeEE21deadlineMonitorThreadEv+0x3b9) [0x6ff469]
mongo() [0x792b79]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7f514f237e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f514e54accd]
Mon Jun 17 10:46:14 EEST 2013
*****************************************************
Mon Jun 17 10:46:15 EEST 2013
MongoDB shell version: 2.4.4
connecting to: hingo-sputnik:27000/rstest
bye
Mon Jun 17 10:46:15 EEST 2013
*****************************************************
Mon Jun 17 10:46:16 EEST 2013
MongoDB shell version: 2.4.4
connecting to: hingo-sputnik:27000/rstest
bye
Mon Jun 17 10:46:17 EEST 2013