Wed Mar 23 10:02:21 [conn1032] autosplitted benchmark.object_interaction shard: ns:benchmark.object_interaction at: shard0000:xx.xx.xx.xxx:10000 lastmod: 2|1 min: { _id: ObjectId('4d8a26da3087913510000157') } max: { _id: ObjectId('4d8a278c30879147420dbb9f') } on: { _id: MaxKey }(splitThreshold 15099494)
Wed Mar 23 10:02:21 [conn1032] moving chunk (auto): ns:benchmark.object_interaction at: shard0000:xx.xx.xx.xxx:10000 lastmod: 2|3 min: { _id: ObjectId('4d8a278c30879147420dbb9f') } max: { _id: MaxKey } to: shard0002:xx.xx.xx.xxx:10002 #objects: 1
Wed Mar 23 10:02:21 [conn1032] moving chunk ns: benchmark.object_interaction moving ( ns:benchmark.object_interaction at: shard0000:xx.xx.xx.xxx:10000 lastmod: 2|3 min: { _id: ObjectId('4d8a278c30879147420dbb9f') } max: { _id: MaxKey }) shard0000:xx.xx.xx.xxx:10000 -> shard0002:xx.xx.xx.xxx:10002
Wed Mar 23 10:02:21 [conn970] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:21 [conn803] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a27233087913ebc04939e') }, max: { _id: MaxKey }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a278c30879147420dbb9a') } ], shardId: "benchmark.object-_id_ObjectId('4d8a27233087913ebc04939e')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8267'), when: new Date(1300899741897), who: "ch-db1.xxx.xx:1300899624:853419240:conn326:1857948162", why: "split-{ _id: ObjectId('4d8a27233087913ebc04939e') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:21 [conn677] autosplitted benchmark.object shard: ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a278c30879147420dbb9a') } on: { _id: MaxKey }(splitThreshold 15099494)
Wed Mar 23 10:02:21 [conn677] moving chunk (auto): ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a278c30879147420dbb9a') } max: { _id: MaxKey } to: shard0002:xx.xx.xx.xxx:10002 #objects: 1
Wed Mar 23 10:02:21 [conn677] moving chunk ns: benchmark.object moving ( ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a278c30879147420dbb9a') } max: { _id: MaxKey }) shard0000:xx.xx.xx.xxx:10000 -> shard0002:xx.xx.xx.xxx:10002
Wed Mar 23 10:02:21 [conn1252] autosplitted benchmark.owner shard: ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a278c30879147420dbb99') } on: { _id: MaxKey }(splitThreshold 15099494)
Wed Mar 23 10:02:21 [conn677] Assertion: 10412:moveAndCommit failed: { assertion: "assertion s/d_migrate.cpp:255", errmsg: "db assertion failure", ok: 0.0 }
0x522c19 0x61342c 0x61b5f2 0x63ac42 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
 /usr/bin/mongos(_ZN5mongo11msgassertedEiPKc+0x129) [0x522c19]
 /usr/bin/mongos(_ZN5mongo5Chunk12moveIfShouldEN5boost10shared_ptrIS0_EE+0x5bc) [0x61342c]
 /usr/bin/mongos(_ZN5mongo5Chunk13splitIfShouldEl+0x302) [0x61b5f2]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x6f2) [0x63ac42]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260) [0x63d6f0]
 /usr/bin/mongos(_ZN5mongo7Request7processEi+0x172) [0x667c42]
 /usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141) [0x67ae11]
 /usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272) [0x5817a2]
 /usr/bin/mongos(thread_proxy+0x80) [0x6a22b0]
 /lib64/libpthread.so.0 [0x3d4260673d]
 /lib64/libc.so.6(clone+0x6d) [0x3d41ed3f6d]
Wed Mar 23 10:02:21 [conn677] warning: could have autosplit on collection: benchmark.object but: moveAndCommit failed: { assertion: "assertion s/d_migrate.cpp:255", errmsg: "db assertion failure", ok: 0.0 }
Wed Mar 23 10:02:21 [conn1252] moving chunk (auto): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey } to: shard0002:xx.xx.xx.xxx:10002 #objects: 1
Wed Mar 23 10:02:21 [conn1252] moving chunk ns: benchmark.owner moving ( ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }) shard0000:xx.xx.xx.xxx:10000 -> shard0002:xx.xx.xx.xxx:10002
Wed Mar 23 10:02:21 [conn858] autosplitted benchmark.object shard: ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } on: { _id: ObjectId('4d8a278c30879147420dbb9a') }(splitThreshold 15099494)
Wed Mar 23 10:02:21 [conn1252] Assertion: 10412:moveAndCommit failed: { assertion: "assertion s/d_migrate.cpp:255", errmsg: "db assertion failure", ok: 0.0 }
0x522c19 0x61342c 0x61b5f2 0x63ac42 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
Wed Mar 23 10:02:21 [conn1123] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
 /usr/bin/mongos(_ZN5mongo11msgassertedEiPKc+0x129) [0x522c19]
 /usr/bin/mongos(_ZN5mongo5Chunk12moveIfShouldEN5boost10shared_ptrIS0_EE+0x5bc) [0x61342c]
 /usr/bin/mongos(_ZN5mongo5Chunk13splitIfShouldEl+0x302) [0x61b5f2]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x6f2) [0x63ac42]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260) [0x63d6f0]
 /usr/bin/mongos(_ZN5mongo7Request7processEi+0x172) [0x667c42]
 /usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141) [0x67ae11]
 /usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272) [0x5817a2]
 /usr/bin/mongos(thread_proxy+0x80) [0x6a22b0]
 /lib64/libpthread.so.0 [0x3d4260673d]
 /lib64/libc.so.6(clone+0x6d) [0x3d41ed3f6d]
Wed Mar 23 10:02:21 [conn1252] warning: could have autosplit on collection: benchmark.owner but: moveAndCommit failed: { assertion: "assertion s/d_migrate.cpp:255", errmsg: "db assertion failure", ok: 0.0 }
Wed Mar 23 10:02:21 [conn1212] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:21 [conn1212] end connection xx.xx.xx.xxx:44459
Wed Mar 23 10:02:21 [conn1429] warning: splitChunk failed - cmd: { splitChunk: "benchmark.owner", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a27243087913ebc0493dc') }, max: { _id: ObjectId('4d8a278c30879147420dbb99') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a275d3087914742025a1e') } ], shardId: "benchmark.owner-_id_ObjectId('4d8a27243087913ebc0493dc')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.owner", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f826c'), when: new Date(1300899741925), who: "ch-db1.xxx.xx:1300899624:853419240:conn923:1594877262", why: "split-{ _id: ObjectId('4d8a27243087913ebc0493dc') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:21 [conn1222] autosplitted benchmark.owner shard: ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025e74') } on: { _id: ObjectId('4d8a278c30879147420dbb99') }(splitThreshold 15099494)
Wed Mar 23 10:02:21 [conn803] end connection xx.xx.xx.xxx:44050
Wed Mar 23 10:02:21 [conn621] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:21 [conn621] end connection xx.xx.xx.xxx:43867
Wed Mar 23 10:02:21 [conn1262] autosplitted benchmark.owner shard: ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } on: { _id: ObjectId('4d8a275d3087914742025e74') }(splitThreshold 15099494)
Wed Mar 23 10:02:21 [conn1262] end connection xx.xx.xx.xxx:44509
Wed Mar 23 10:02:21 [conn1373] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:21 [conn1330] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:21 [conn1507] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:21 [conn1507] end connection xx.xx.xx.xxx:44757
Wed Mar 23 10:02:21 [conn1330] end connection xx.xx.xx.xxx:44577
Wed Mar 23 10:02:21 [conn1373] end connection xx.xx.xx.xxx:44623
Wed Mar 23 10:02:22 [conn1306] warning: splitChunk failed - cmd: { splitChunk: "benchmark.owner", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a27243087913ebc0493dc') }, max: { _id: ObjectId('4d8a275d3087914742025a1e') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a275d3087914742026f97') } ], shardId: "benchmark.owner-_id_ObjectId('4d8a27243087913ebc0493dc')", configdb: "xx.xx.xx.xxx:27019" } result: { assertion: "can split { _id: ObjectId('4d8a27243087913ebc0493dc') } -> { _id: Obje...", assertionCode: 13593, errmsg: "db assertion failure", ok: 0.0 }
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn873] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn1165] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn873] end connection xx.xx.xx.xxx:44120
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn769] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn994] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn769] end connection xx.xx.xx.xxx:44016
Wed Mar 23 10:02:22 [conn994] end connection xx.xx.xx.xxx:44241
Wed Mar 23 10:02:22 [conn1502] warning: splitChunk failed - cmd: { splitChunk: "benchmark.owner", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a27243087913ebc0493dc') }, max: { _id: ObjectId('4d8a275d3087914742025a1e') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a275d3087914742025d17') } ], shardId: "benchmark.owner-_id_ObjectId('4d8a27243087913ebc0493dc')", configdb: "xx.xx.xx.xxx:27019" } result: { currMin: { _id: ObjectId('4d8a27243087913ebc0493dc') }, currMax: { _id: ObjectId('4d8a275d3087914742026f97') }, requestedMin: { _id: ObjectId('4d8a27243087913ebc0493dc') }, requestedMax: { _id: ObjectId('4d8a275d3087914742025a1e') }, errmsg: "chunk boundaries are outdated (likely a split occurred)", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1175] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1175] end connection xx.xx.xx.xxx:44422
Wed Mar 23 10:02:22 [conn644] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn942] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1357] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1105] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1306] Assertion: 13282:Couldn't load a valid config for benchmark.owner after 3 attempts. Please try again.
0x522c19 0x611525 0x611cff 0x619b03 0x61af5f 0x61b485 0x63ac42 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
 /usr/bin/mongos(_ZN5mongo11msgassertedEiPKc+0x129) [0x522c19]
 /usr/bin/mongos(_ZN5mongo12ChunkManager14_reload_inlockEv+0x1b5) [0x611525]
 /usr/bin/mongos(_ZN5mongo12ChunkManager7_reloadEv+0x2f) [0x611cff]
 /usr/bin/mongos(_ZN5mongo5Chunk10multiSplitERKSt6vectorINS_7BSONObjESaIS2_EERS2_+0xb43) [0x619b03]
 /usr/bin/mongos(_ZN5mongo5Chunk11singleSplitEbRNS_7BSONObjE+0x4cf) [0x61af5f]
 /usr/bin/mongos(_ZN5mongo5Chunk13splitIfShouldEl+0x195) [0x61b485]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x6f2) [0x63ac42]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260) [0x63d6f0]
 /usr/bin/mongos(_ZN5mongo7Request7processEi+0x172) [0x667c42]
 /usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141) [0x67ae11]
 /usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272) [0x5817a2]
 /usr/bin/mongos(thread_proxy+0x80) [0x6a22b0]
 /lib64/libpthread.so.0 [0x3d4260673d]
 /lib64/libc.so.6(clone+0x6d) [0x3d41ed3f6d]
Wed Mar 23 10:02:22 [conn644] end connection xx.xx.xx.xxx:43891
Wed Mar 23 10:02:22 [conn1306] warning: could have autosplit on collection: benchmark.owner but: Couldn't load a valid config for benchmark.owner after 3 attempts. Please try again.
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1502] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn1508] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1511] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn996] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1335] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn678] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn706] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1502] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn706] end connection xx.xx.xx.xxx:43953
Wed Mar 23 10:02:22 [conn678] end connection xx.xx.xx.xxx:43925
Wed Mar 23 10:02:22 [conn673] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn1341] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn673] end connection xx.xx.xx.xxx:43920
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1502] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1502] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn1017] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1263] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn650] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn1156] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn860] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a27233087913ebc04939e') }, max: { _id: ObjectId('4d8a275d308791474202479c') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a275d3087914742024bb8') } ], shardId: "benchmark.object-_id_ObjectId('4d8a27233087913ebc04939e')", configdb: "xx.xx.xx.xxx:27019" } result: { assertion: "can split { _id: ObjectId('4d8a27233087913ebc04939e') } -> { _id: Obje...", assertionCode: 13593, errmsg: "db assertion failure", ok: 0.0 }
it->second->toString(): ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742024bb8') } max: { _id: ObjectId('4d8a275d308791474202479c') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742024bb8') }
last->second->getMax(): { _id: ObjectId('4d8a27233087913ebc04939e') }
Wed Mar 23 10:02:22 [conn860] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26d93087913510000000') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|5 min: { _id: ObjectId('4d8a26d93087913510000000') } max: { _id: ObjectId('4d8a27133087913ebc0008c1') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a27133087913ebc0008c1') } max: { _id: ObjectId('4d8a27233087913ebc04939e') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742024bb8') } max: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d3087914742024bb8') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d308791474202479c') } max: { _id: ObjectId('4d8a278c30879147420dbb9a') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a278c30879147420dbb9a') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn1033] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn921] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1049] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn965] warning: splitChunk failed - cmd: { splitChunk: "benchmark.object_interaction", keyPattern: { _id: 1.0 }, min: { _id: ObjectId('4d8a26da3087913510000157') }, max: { _id: ObjectId('4d8a278c30879147420dbb9f') }, from: "xx.xx.xx.xxx:10000", splitKeys: [ { _id: ObjectId('4d8a27133087913ebc0003d6') } ], shardId: "benchmark.object_interaction-_id_ObjectId('4d8a26da3087913510000157')", configdb: "xx.xx.xx.xxx:27019" } result: { who: { _id: "benchmark.object_interaction", process: "ch-db1.xxx.xx:1300899624:853419240", state: 1, ts: ObjectId('4d8a279d2d0638eec26f8266'), when: new Date(1300899741893), who: "ch-db1.xxx.xx:1300899624:853419240:conn657:900415947", why: "migrate-{ _id: ObjectId('4d8a278c30879147420dbb9f') }" }, errmsg: "the collection's metadata lock is taken", ok: 0.0 }
Wed Mar 23 10:02:22 [conn1387] want to split chunk, but can't find split point chunk ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742025a1e') } got: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn653] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn805] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn804] want to split chunk, but can't find split point chunk ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|4 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d308791474202479c') } got: { _id: ObjectId('4d8a275d308791474202479c') }
it->second->toString(): ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742024bb8') } max: { _id: ObjectId('4d8a275d308791474202479c') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742024bb8') }
last->second->getMax(): { _id: ObjectId('4d8a27233087913ebc04939e') }
Wed Mar 23 10:02:22 [conn860] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26d93087913510000000') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|5 min: { _id: ObjectId('4d8a26d93087913510000000') } max: { _id: ObjectId('4d8a27133087913ebc0008c1') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a27133087913ebc0008c1') } max: { _id: ObjectId('4d8a27233087913ebc04939e') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742024bb8') } max: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d3087914742024bb8') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d308791474202479c') } max: { _id: ObjectId('4d8a278c30879147420dbb9a') }
Wed Mar 23 10:02:22 [conn860] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a278c30879147420dbb9a') } max: { _id: MaxKey }
Wed Mar 23 10:02:22 [conn1502] Assertion: 13282:Couldn't load a valid config for benchmark.owner after 3 attempts. Please try again.
0x522c19 0x611525 0x611cff 0x619b03 0x61af5f 0x61b485 0x63ac42 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
 /usr/bin/mongos(_ZN5mongo11msgassertedEiPKc+0x129) [0x522c19]
 /usr/bin/mongos(_ZN5mongo12ChunkManager14_reload_inlockEv+0x1b5) [0x611525]
 /usr/bin/mongos(_ZN5mongo12ChunkManager7_reloadEv+0x2f) [0x611cff]
 /usr/bin/mongos(_ZN5mongo5Chunk10multiSplitERKSt6vectorINS_7BSONObjESaIS2_EERS2_+0xb43) [0x619b03]
 /usr/bin/mongos(_ZN5mongo5Chunk11singleSplitEbRNS_7BSONObjE+0x4cf) [0x61af5f]
 /usr/bin/mongos(_ZN5mongo5Chunk13splitIfShouldEl+0x195) [0x61b485]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x6f2) [0x63ac42]
 /usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260) [0x63d6f0]
 /usr/bin/mongos(_ZN5mongo7Request7processEi+0x172) [0x667c42]
 /usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141) [0x67ae11]
 /usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272) [0x5817a2]
 /usr/bin/mongos(thread_proxy+0x80) [0x6a22b0]
 /lib64/libpthread.so.0 [0x3d4260673d]
 /lib64/libc.so.6(clone+0x6d) [0x3d41ed3f6d]
Wed Mar 23 10:02:22 [conn1502] warning: could have autosplit on collection: benchmark.owner but: Couldn't load a valid config for benchmark.owner after 3 attempts. Please try again.
foo: { _id: ObjectId('4d8a275d3087914742025a1e') }
*c: ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
key: { _id: ObjectId('4d8a275c30879147420217ba') }
Wed Mar 23 10:02:22 [conn1306] ChunkManager: couldn't find chunk for: { _id: ObjectId('4d8a276f30879147420798d6') } going to retry
foo: { _id: MaxKey }
*c: Wed Mar 23 10:02:22 [conn1508] ChunkManager: couldn't find chunk for: { _id: ObjectId('4d8a2770308791474207f3b8') } going to retry
Received signal 11
Wed Mar 23 10:02:22 [conn1387] ChunkManager: couldn't find chunk for: { _id: ObjectId('4d8a277a30879147420aecc4') } going to retry
Backtrace: nal 11
Backtrace: nal 11
Backtrace: Backtrace: Backtrace: ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
key: { _id: ObjectId('4d8a275c3087914742021da6') }
0x5319a5 0x3d41e302d0 0x4b6105 0x5319a5 0x610d38 0x61144b 0x6118cc 0x63aad3 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
3d41ed30x3d41e302d0 0x4b6113 00x3d41e302d0 0x4b6113 0x6116d3 0x63aad3 0x6116d3 0x63aad3 0x63d6f0 0x667c42 0x67ae11 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 

3d41ed3it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x5319a5]
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1306] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x5319a5]
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
/lib64/libc.so.6[0x3d41e302d0]
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1306] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x5319a5]
/usr/bin/mongos(_ZNK5mongo7BSONObj9woCompareERKS0_S2_b+0x15)[0x4b6105]
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
/lib64/libc.so.6[0x3d41e302d0]
Wed Mar 23 10:02:22 [conn1387] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
{ _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1508] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1508] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
/usr/bin/mongos(_ZNK5mongo7BSONObj9woCompareERKS0_S2_b+0x23)[0x4b6113]
Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Received signal 11
Backtrace: Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1387] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
it->second->toString(): ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742026f97') }
last->second->getMax(): { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1511] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26da3087913510000316') }
/usr/bin/mongos(_ZNSt3mapIN5mongo7BSONObjEN5boost10shared_ptrINS0_5ChunkEEENS0_10BSONObjCmpESaISt4pairIKS1_S5_EEEixERS8_+0x50)[0x61e750]
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|3 min: { _id: ObjectId('4d8a26da3087913510000316') } max: { _id: ObjectId('4d8a27243087913ebc0493dc') }
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742026f97') } max: { _id: ObjectId('4d8a275d3087914742025a1e') }
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d3087914742025a1e') } max: { _id: ObjectId('4d8a275d3087914742025e74') }
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27243087913ebc0493dc') } max: { _id: ObjectId('4d8a275d3087914742026f97') }
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a275d3087914742025e74') } max: { _id: ObjectId('4d8a278c30879147420dbb99') }
Wed Mar 23 10:02:22 [conn1511] ns:benchmark.owner at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a278c30879147420dbb99') } max: { _id: MaxKey }
/usr/bin/mongos(_ZN5mongo12ChunkManager5_loadEv+0x5e8)[0x610d38]
/usr/bin/mongos(_ZN5mongo12ChunkManager9findChunkERKNS_7BSONObjEb+0xe3)[0x6116d3]
it->second->toString(): ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742024bb8') } max: { _id: ObjectId('4d8a275d308791474202479c') }
it->second->getMin(): { _id: ObjectId('4d8a275d3087914742024bb8') }
/lib64/libc.so.6[0x3d41e302d0]
last->second->getMax(): { _id: ObjectId('4d8a27233087913ebc04939e') }
Wed Mar 23 10:02:22 [conn1335] ChunkManager::_isValid failed: it->second->getMin() == last->second->getMax()
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|1 min: { _id: MinKey } max: { _id: ObjectId('4d8a26d93087913510000000') }
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|5 min: { _id: ObjectId('4d8a26d93087913510000000') } max: { _id: ObjectId('4d8a27133087913ebc0008c1') }
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|6 min: { _id: ObjectId('4d8a27133087913ebc0008c1') } max: { _id: ObjectId('4d8a27233087913ebc04939e') }
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|12 min: { _id: ObjectId('4d8a275d3087914742024bb8') } max: { _id: ObjectId('4d8a275d308791474202479c') }
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|11 min: { _id: ObjectId('4d8a27233087913ebc04939e') } max: { _id: ObjectId('4d8a275d3087914742024bb8') }
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|10 min: { _id: ObjectId('4d8a275d308791474202479c') } max: { _id: ObjectId('4d8a278c30879147420dbb9a') }
/usr/bin/mongos(_ZN5mongo12ChunkManager14_reload_inlockEv+0xdb)[0x61144b]
Wed Mar 23 10:02:22 [conn1335] ns:benchmark.object at: shard0000:xx.xx.xx.xxx:10000 lastmod: 1|8 min: { _id: ObjectId('4d8a278c30879147420dbb9a') } max: { _id: MaxKey }
/usr/bin/mongos(_ZN5mongo12ChunkManager9findChunkERKNS_7BSONObjEb+0x2dc)[0x6118cc]
/usr/bin/mongos(_ZNK5mongo7BSONObj9woCompareERKS0_S2_b+0x23)[0x4b6113]
/usr/bin/mongos(_ZN5mongo12ChunkManager9findChunkERKNS_7BSONObjEb+0xe3)[0x6116d3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x583)[0x63aad3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x583)[0x63aad3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63d6f0]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x583)[0x63aad3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63d6f0]
/usr/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x667c42]
/usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141)[0x67ae11]
/usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272)[0x5817a2]
/usr/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x667c42]
/usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141)[0x67ae11]
/usr/bin/mongos(thread_proxy+0x80)[0x6a22b0]
/usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272)[0x5817a2]
/usr/bin/mongos(thread_proxy+0x80)[0x6a22b0]
/lib64/libpthread.so.0[0x3d4260673d]
/lib64/libc.so.6(clone+0x6d)[0x3d41ed3f6d]
===
/lib64/libpthread.so.0[0x3d4260673d]
/lib64/libc.so.6(clone+0x6d)[0x3d41ed3f6d]
===
Received signal 6
Backtrace: /usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63d6f0]
/usr/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x667c42]
/usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141)[0x67ae11]
/usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272)[0x5817a2]
0x5319a5 0x3d41e302d0 0x3d41e30265 0x3d41e31d10 0x3d41e6a84b 0x3d41e72405 0x3d41e7276b 0x5355d2 0x5374a0 0x3d41e333a5 0x531a0b 0x3d41e302d0 0x4b6113 0x6116d3 0x63aad3 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x5319a5]
/lib64/libc.so.6[0x3d41e302d0]
/lib64/libc.so.6(gsignal+0x35)[0x3d41e30265]
/lib64/libc.so.6(abort+0x110)[0x3d41e31d10]
/lib64/libc.so.6[0x3d41e6a84b]
/lib64/libc.so.6[0x3d41e72405]
/lib64/libc.so.6(cfree+0x4b)[0x3d41e7276b]
/usr/bin/mongos(_ZN5mongo11PoolForHostD1Ev+0x22)[0x5355d2]
/usr/bin/mongos(_ZN5mongo16DBConnectionPoolD1Ev+0x80)[0x5374a0]
/lib64/libc.so.6(exit+0xe5)[0x3d41e333a5]
/usr/bin/mongos[0x531a0b]
/lib64/libc.so.6[0x3d41e302d0]
/usr/bin/mongos(_ZNK5mongo7BSONObj9woCompareERKS0_S2_b+0x23)[0x4b6113]
/usr/bin/mongos(_ZN5mongo12ChunkManager9findChunkERKNS_7BSONObjEb+0xe3)[0x6116d3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x583)[0x63aad3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63d6f0]
/usr/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x667c42]
/usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141)[0x67ae11]
/usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272)[0x5817a2]
/usr/bin/mongos(thread_proxy+0x80)[0x6a22b0]
===
/usr/bin/mongos(thread_proxy+0x80)[0x6a22b0]
/lib64/libpthread.so.0[0x3d4260673d]
/lib64/libc.so.6(clone+0x6d)[0x3d41ed3f6d]
===
0x5319a5 0x3d41e302d0 0x62113c 0x60a835 0x62124d 0x4abdaa 0x60678d 0x61154c 0x6119da 0x63aad3 0x63d6f0 0x667c42 0x67ae11 0x5817a2 0x6a22b0 0x3d4260673d 0x3d41ed3f6d 
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x5319a5]
/lib64/libc.so.6[0x3d41e302d0]
/usr/bin/mongos(_ZN5mongolsERSoRKNS_7BSONObjE+0x1c)[0x62113c]
/usr/bin/mongos(_ZNK5mongo5Chunk8toStringEv+0x195)[0x60a835]
/usr/bin/mongos(_ZNK5mongo14LazyStringImplINS_5ChunkEE3valEv+0xd)[0x62124d]
/usr/bin/mongos(_ZN5mongo9LogstreamlsERKNS_10LazyStringE+0x1a)[0x4abdaa]
/usr/bin/mongos(_ZNK5mongo12ChunkManager12_printChunksEv+0x6d)[0x60678d]
/usr/bin/mongos(_ZN5mongo12ChunkManager14_reload_inlockEv+0x1dc)[0x61154c]
/usr/bin/mongos(_ZN5mongo12ChunkManager9findChunkERKNS_7BSONObjEb+0x3ea)[0x6119da]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12ChunkManagerEEE+0x583)[0x63aad3]
/usr/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63d6f0]
/usr/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x667c42]
/usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x141)[0x67ae11]
/usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x272)[0x5817a2]
/usr/bin/mongos(thread_proxy+0x80)[0x6a22b0]
/lib64/libpthread.so.0[0x3d4260673d]
/lib64/libc.so.6(clone+0x6d)[0x3d41ed3f6d]
===
Wed Mar 23 10:05:09 /usr/bin/mongos db version v1.8.0, pdfile version 4.5 starting (--help for usage)
Wed Mar 23 10:05:09 git version: 9c28b1d608df0ed6ebe791f63682370082da41c0
Wed Mar 23 10:05:09 build sys info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Wed Mar 23 10:05:09 [websvr] web admin interface listening on port 28017
Wed Mar 23 10:05:09 [mongosMain] waiting for connections on port 27017
Wed Mar 23 10:05:09 [Balancer] about to contact config servers and shards
Wed Mar 23 10:05:09 [Balancer] config servers and shards contacted successfully
Wed Mar 23 10:05:09 [Balancer] balancer id: ch-db1.xxx.xx:27017 started at Mar 23 10:05:09
Wed Mar 23 10:05:09 [LockPinger] creating dist lock ping thread for: xx.xx.xx.xxx:27019
Wed Mar 23 10:05:14 [mongosMain] connection accepted from 127.0.0.1:42606 #1
Wed Mar 23 10:05:17 [conn1] creating WriteBackListener for: xx.xx.xx.xxx:10000
Wed Mar 23 10:05:17 [conn1] creating WriteBackListener for: xx.xx.xx.xxx:10001
Wed Mar 23 10:05:17 [conn1] creating WriteBackListener for: xx.xx.xx.xxx:10002
Wed Mar 23 10:05:17 [conn1] creating WriteBackListener for: xx.xx.xx.xxx:27019
W