-
Type: Bug
-
Resolution: Won't Fix
-
Priority: Minor - P4
-
None
-
Affects Version/s: None
-
Component/s: HTTP Console
-
None
-
ALL
-
The HTTP console UI hits an assert in db/stats/snapshots.cpp if the system clock has been set back in time recently. Including when the clock has been set automatically by ntpd. This results in the below messages, and an HTTP 500 error from the HTTP console:
mongodb 2.0.4
Tue Nov 25 14:44:17 [websvr] Assertion failure _newer._created > _older._created db/stats/snapshots.cpp 39
mongodb 2.4.9
Tue Nov 25 13:18:04.743 [websvr] Assertion failure _newer._created > _older._created src/mongo/db/stats/snapshots.cpp 40 0x981b16 0x945a30 0x886673 0x886c53 0x888010 0x6ce7d9 0x6d367d 0x96eed5 0x96ab25 0x6ce924 0x7f049c6d0a4a 0x7f049cf1b182 0x7f049b2e6fbd /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0x981b16] /usr/bin/mongod(_ZN5mongo12verifyFailedEPKcS1_j+0xc0) [0x945a30] /usr/bin/mongod() [0x886673] /usr/bin/mongod(_ZN5mongo9Snapshots18outputLockInfoHTMLERSt18basic_stringstreamIcSt11char_traitsIcESaIcEE+0x133) [0x886c53] /usr/bin/mongod(_ZN5mongo15WriteLockStatus3runERSt18basic_stringstreamIcSt11char_traitsIcESaIcEE+0x20) [0x888010] /usr/bin/mongod(_ZN5mongo15WebStatusPlugin6runAllERSt18basic_stringstreamIcSt11char_traitsIcESaIcEE+0xd9) [0x6ce7d9] /usr/bin/mongod(_ZN5mongo11DbWebServer9doRequestEPKcSsRSsRiRSt6vectorISsSaISsEERKNS_8SockAddrE+0x86d) [0x6d367d] /usr/bin/mongod(_ZN5mongo13MiniWebServer8acceptedEN5boost10shared_ptrINS_6SocketEEEx+0x165) [0x96eed5] /usr/bin/mongod(_ZN5mongo8Listener13initAndListenEv+0x585) [0x96ab25] /usr/bin/mongod(_ZN5mongo15webServerThreadEPKNS_11AdminAccessE+0x94) [0x6ce924] /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0(+0xba4a) [0x7f049c6d0a4a] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f049cf1b182] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f049b2e6fbd]
This is probably the same underlying issue as SERVER-7788. However, I think it should get a second look. This makes the console not reliable for automated monitoring. Also, regular users will see the assertion failure in their browser and get no indication why.
My plan to avoid hitting this again is to enable the REST API with --rest, and have our monitoring tool use that instead.
My thought is that since this only seems to affect the DbTopStatus and WriteLockStatus objects, can the failure be caught in there, and not prevent the entire HTTP console from loading?