Loading...

XML

Word

Printable

JSON

Type: Bug
Resolution: Duplicate
Priority: Major - P3
Fix Version/s: None
Affects Version/s: 2.4.6
Component/s: Querying, Sharding
Labels:
- balancing
- chunk
- query
- shardVersion
Environment:
Linux 64 Bit

Operating System:
Linux

We are facing the above mentioned bug for a long time already.
The first bug report has been opened in Nov 2011 already (~~SERVER-4185~~) and should have been fixed in v2.1.0 but the bug survived. That's why 10gen suggested us to open a new bug report (~~SERVER-8872~~) which however has been flagged as "duplicate" (hence "resolved") to ~~SERVER-5752~~ which in turn "needs triage" since May 03 2012.

The bug is a big pain for us because long(er) running queries might break and just return an error. We are very worried that the error will happen more frequently because our sharded mongodb grows continuously 50%-100% per year. Right now we have 39 mongod's = 13 replicasets (shards) each one consisting of 3 mongod's. The bigger the db, the more chunk moves will happen, thus we will be more exposed to the above mentionned bug.

Can you please investigate the bug and eventually fix it definitively, or at least describe a work around?
Thank you!

Please see a log snippet from a mongod where the bug caused round about 40 queries to die:

Mon Oct 21 16:16:02.568 [conn2428544] end connection 172.16.64.143:46345 (6250 connections now open)
Mon Oct 21 16:16:02.864 [conn2428419] info DFM::findAll(): extent 78:447fa000 was empty, skipping ahead. ns:offerStore.offer
Mon Oct 21 16:16:03.727 [conn2428542] end connection 172.16.64.143:46335 (6249 connections now open)
Mon Oct 21 16:16:04.963 [conn2428239] getmore offerStore.offer query: { missingSince: null, shopId: 62184 } cursorid:3771051008520488581 ntoreturn:0 keyUpdates:0 numYields: 804 locks(micros) r:5651276 nreturned:72315 reslen:4194290 41852ms
Mon Oct 21 16:16:05.672 [migrateThread] Waiting for replication to catch up before entering critical section
Mon Oct 21 16:16:05.672 [migrateThread] migrate commit succeeded flushing to secondaries for 'offerStore.offer' { _id: 1027094894 } -> { _id: 1027214746 }
Mon Oct 21 16:16:06.064 [initandlisten] connection accepted from 172.16.66.1:58352 #2428950 (6250 connections now open)
Mon Oct 21 16:16:06.066 [conn2428950]  authenticate db: local { authenticate: 1, nonce: "61f9f98fe44678aa", user: "__system", key: "9a0cb2cc7d8a6a1c2f2126d7cff39176" }
Mon Oct 21 16:16:06.066 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.067 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.069 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.070 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.071 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.072 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.073 [conn2428950] Waiting for commit to finish
Mon Oct 21 16:16:06.073 [migrateThread] migrate commit succeeded flushing to secondaries for 'offerStore.offer' { _id: 1027094894 } -> { _id: 1027214746 }
Mon Oct 21 16:16:06.073 [migrateThread] about to log metadata event: { _id: "s132-2013-10-21T14:16:06-526537260b545bae51c2c43c", server: "s132", clientAddr: ":27017", time: new Date(1382364966073), what: "moveChunk.to", ns: "offerStore.offer", details: { min: { _id: 1027094894 }, max: { _id: 1027214746 }, step1 of 5: 441, step2 of 5: 28, step3 of 5: 348628, step4 of 5: 42855, step5 of 5: 401 } }
Mon Oct 21 16:16:06.073 [migrateThread] SyncClusterConnection connecting to [sx210:20020]
Mon Oct 21 16:16:06.082 [migrateThread] SyncClusterConnection connecting to [sx176:20020]
Mon Oct 21 16:16:06.087 [migrateThread] SyncClusterConnection connecting to [sx177:20020]
Mon Oct 21 16:16:06.312 [conn2422064] SyncClusterConnection connecting to [sx210:20020]
Mon Oct 21 16:16:06.312 [conn2425307] SyncClusterConnection connecting to [sx210:20020]
Mon Oct 21 16:16:06.314 [conn2422064] SyncClusterConnection connecting to [sx176:20020]
Mon Oct 21 16:16:06.315 [conn2425307] SyncClusterConnection connecting to [sx176:20020]
Mon Oct 21 16:16:06.321 [conn2422064] SyncClusterConnection connecting to [sx177:20020]
Mon Oct 21 16:16:06.323 [conn2425307] SyncClusterConnection connecting to [sx177:20020]
Mon Oct 21 16:16:06.386 [conn2428804] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ _id: 1726897817 }
Mon Oct 21 16:16:06.386 [conn2421423] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ exportFP.ANP: { $exists: 1 }, shopId: 223581 }
Mon Oct 21 16:16:06.386 [conn2428804]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.386 [conn2426350] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ exportFP.MAPPED: { $exists: 1 }, shopId: 223581 }
Mon Oct 21 16:16:06.386 [conn2421423]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.386 [conn2426350]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.386 [conn2424327] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ exportFP.ANP: { $exists: 1 }, shopId: 10094 }
Mon Oct 21 16:16:06.386 [conn2426350] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2424327]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.386 [conn2426131] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ asins: "B0090OKDB8" }
Mon Oct 21 16:16:06.386 [conn2424327] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2428804] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2010109] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ $query: { bokey: { $in: [ "12041:SOjBKCmN6WAZTOecjcxURA" ] } }, $readPreference: { mode: "primary" } }
Mon Oct 21 16:16:06.386 [conn2428054] Count with ns: offerStore.offer and query: { shopId: 255697 } failed with exception: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code: 13388
Mon Oct 21 16:16:06.386 [conn2426186] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ exportFP.ANP: { $exists: 1 }, shopId: 275982 }
Mon Oct 21 16:16:06.386 [conn2425736] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ exportFP.IDEALO: { $exists: 1 }, shopId: 244324 }
Mon Oct 21 16:16:06.386 [conn2425736]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.386 [conn2426131] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2425736] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.387 [conn2425736] query offerStore.offer query: { exportFP.IDEALO: { $exists: 1 }, shopId: 244324 } ntoreturn:1 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 248 locks(micros) r:969282 reslen:569 10866ms
Mon Oct 21 16:16:06.387 [conn2426131] query offerStore.offer query: { asins: "B0090OKDB8" } ntoreturn:0 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 23449 locks(micros) r:498214965 reslen:569 1009623ms
Mon Oct 21 16:16:06.386 [conn2421423] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2010109]  ntoskip:0 ntoreturn:500
Mon Oct 21 16:16:06.387 [conn2010109] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2427564] Count with ns: offerStore.offer and query: { missingSince: { $ne: null }, shopId: 9701 } failed with exception: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code: 13388
Mon Oct 21 16:16:06.386 [conn2426186]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.387 [conn2426186] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2428080] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ sku: "B0090OKDB8" }
Mon Oct 21 16:16:06.386 [conn2425933] assertion 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ns:offerStore.offer query:{ exportFP.ANP: { $exists: 1 }, shopId: 216050 }
Mon Oct 21 16:16:06.387 [conn2425933]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.387 [conn2425933] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.386 [conn2424327] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 10094 } ntoreturn:1 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 1134 locks(micros) r:7076921 reslen:569 59706ms
Mon Oct 21 16:16:06.386 [conn2426350] query offerStore.offer query: { exportFP.MAPPED: { $exists: 1 }, shopId: 223581 } ntoreturn:1 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 1988 locks(micros) r:8773375 reslen:569 117129ms
Mon Oct 21 16:16:06.387 [conn2421423] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 223581 } ntoreturn:1 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 118 locks(micros) r:475493 reslen:569 4715ms
Mon Oct 21 16:16:06.387 [conn2426186] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 275982 } ntoreturn:1 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 1988 locks(micros) r:46702946 reslen:569 108195ms
Mon Oct 21 16:16:06.387 [conn2428080]  ntoskip:0 ntoreturn:-1
Mon Oct 21 16:16:06.387 [conn2428080] stale version detected during query over offerStore.offer : { $err: "[offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match ...", code: 13388, ns: "offerStore.offer", vReceived: Timestamp 22731000|0, vReceivedEpoch: ObjectId('52304df1a7e317e17cd6b0dc'), vWanted: Timestamp 22735000|0, vWantedEpoch: ObjectId('52304df1a7e317e17cd6b0dc') }
Mon Oct 21 16:16:06.387 [conn2428080] query offerStore.offer query: { sku: "B0090OKDB8" } ntoreturn:1 keyUpdates:0 exception: [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send ) code:13388 numYields: 22043 locks(micros) r:452145029 reslen:569 955351ms
Mon Oct 21 16:16:06.387 [conn2427564] command offerStore.$cmd command: { count: "offer", query: { missingSince: { $ne: null }, shopId: 9701 } } ntoreturn:1 keyUpdates:0 numYields: 73850 locks(micros) r:2150582843 reslen:342 2229947ms
Mon Oct 21 16:16:06.386 [conn2428054] command offerStore.$cmd command: { count: "offer", query: { shopId: 255697 } } ntoreturn:1 keyUpdates:0 numYields: 265 locks(micros) r:576799 reslen:342 13896ms
Mon Oct 21 16:16:06.388 [conn2421423] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.388 [conn2426131] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.390 [conn2426350] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.390 [conn2426186] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.390 [conn2425736] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.390 [conn2428080] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.401 [conn2428054] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.402 [conn2424327] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.402 [conn2427564] warning: Caught Assertion while trying to profile query against offerStore.offer: 13388 [offerStore.offer] shard version not ok in Client::Context: version mismatch detected for offerStore.offer, stored major version 22735 does not match received 22731 ( ns : offerStore.offer, received : 22731|0||52304df1a7e317e17cd6b0dc, wanted : 22735|0||52304df1a7e317e17cd6b0dc, send )
Mon Oct 21 16:16:06.599 [conn2428942] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 62184 } cursorid:3771016216249581748 ntoreturn:0 keyUpdates:0 numYields: 272 locks(micros) r:1733994 nreturned:35204 reslen:3767813 10511ms
Mon Oct 21 16:16:06.814 [conn2375767] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 23863 } cursorid:3771071458020254876 ntoreturn:0 keyUpdates:0 numYields: 980 locks(micros) r:5255897 nreturned:6305 reslen:764191 50178ms
Mon Oct 21 16:16:07.897 [conn2380930] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 1140 } cursorid:3771784302268505319 ntoreturn:0 keyUpdates:0 numYields: 3640 locks(micros) r:24278105 nreturned:37331 reslen:4194315 205072ms
Mon Oct 21 16:16:08.237 [conn2428948] end connection 172.16.64.39:38169 (6249 connections now open)
Mon Oct 21 16:16:08.238 [initandlisten] connection accepted from 172.16.64.39:38171 #2428951 (6250 connections now open)
Mon Oct 21 16:16:08.240 [conn2428951]  authenticate db: local { authenticate: 1, nonce: "53ec9359fbbdee1d", user: "__system", key: "4dca396bde91a9cc2948689fd40956c1" }
Mon Oct 21 16:16:10.656 [conn2426186] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 275982 } ntoreturn:1 ntoskip:0 nscanned:152419 keyUpdates:0 numYields: 378 locks(micros) r:5712606 nreturned:0 reslen:20 4254ms
Mon Oct 21 16:16:11.432 [conn2424327] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 10094 } ntoreturn:1 ntoskip:0 nscanned:175801 keyUpdates:0 numYields: 427 locks(micros) r:6650160 nreturned:0 reslen:20 5022ms
Mon Oct 21 16:16:15.005 [conn2426350] query offerStore.offer query: { exportFP.MAPPED: { $exists: 1 }, shopId: 223581 } ntoreturn:1 ntoskip:0 nscanned:380361 keyUpdates:0 numYields: 783 locks(micros) r:11887926 nreturned:0 reslen:20 8600ms
Mon Oct 21 16:16:15.459 [initandlisten] connection accepted from 172.16.64.161:42191 #2428952 (6251 connections now open)
Mon Oct 21 16:16:15.462 [conn2428952]  authenticate db: local { authenticate: 1, nonce: "fb95edebc414f83f", user: "__system", key: "83f11c9bc83ed7dce8d41eebbe1bcce0" }
Mon Oct 21 16:16:15.628 [initandlisten] connection accepted from 172.16.64.161:42197 #2428953 (6252 connections now open)
Mon Oct 21 16:16:15.630 [conn2428953]  authenticate db: local { authenticate: 1, nonce: "5a8cdfed0b069537", user: "__system", key: "3304de203407fe488024accd57d4c8fb" }
Mon Oct 21 16:16:15.634 [initandlisten] connection accepted from 172.16.64.161:42200 #2428954 (6253 connections now open)
Mon Oct 21 16:16:15.636 [conn2428954]  authenticate db: local { authenticate: 1, nonce: "e29d9811eaf59022", user: "__system", key: "d41703b8d0f872c78466c0b590bc9664" }
Mon Oct 21 16:16:16.221 [conn2424118] request split points lookup for chunk offerStore.offer { : 1722508983 } -->> { : 1722540232 }
Mon Oct 21 16:16:16.768 [initandlisten] connection accepted from 172.16.64.161:42211 #2428955 (6254 connections now open)
Mon Oct 21 16:16:16.771 [conn2428955]  authenticate db: local { authenticate: 1, nonce: "220fd9cdb0bda0db", user: "__system", key: "18290950c00f386324340d75fb4b6a1c" }
Mon Oct 21 16:16:17.800 [initandlisten] connection accepted from 172.16.64.161:42216 #2428956 (6255 connections now open)
Mon Oct 21 16:16:17.802 [conn2428956]  authenticate db: local { authenticate: 1, nonce: "b8ae9e81e05fe756", user: "__system", key: "b066dfaa48b09237e4b9a135f4330365" }
Mon Oct 21 16:16:17.850 [initandlisten] connection accepted from 172.16.64.161:42221 #2428957 (6256 connections now open)
Mon Oct 21 16:16:17.851 [conn2428957]  authenticate db: local { authenticate: 1, nonce: "eb4c053cb583f8bd", user: "__system", key: "06d63a829f4e7cf22cbf3d8a3df4b9bb" }
Mon Oct 21 16:16:19.228 [conn2421423] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 223581 } ntoreturn:1 ntoskip:0 nscanned:380361 keyUpdates:0 numYields: 683 locks(micros) r:10995975 nreturned:0 reslen:20 8302ms
Mon Oct 21 16:16:21.733 [conn2426350] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 223581 } ntoreturn:1 ntoskip:0 nscanned:380361 keyUpdates:0 numYields: 471 locks(micros) r:9207415 nreturned:0 reslen:20 6721ms
Mon Oct 21 16:16:22.880 [conn2380930] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 1140 } cursorid:3771784302268505319 ntoreturn:0 keyUpdates:0 numYields: 694 locks(micros) r:3188504 nreturned:8329 reslen:966517 3043ms
Mon Oct 21 16:16:24.987 [initandlisten] connection accepted from 172.16.64.161:42236 #2428958 (6257 connections now open)
Mon Oct 21 16:16:24.990 [conn2428958]  authenticate db: local { authenticate: 1, nonce: "be13b19fcbd6abaf", user: "__system", key: "4f80f5d8b43e71ddf6f47070ebbb5ba5" }
Mon Oct 21 16:16:25.985 [conn2418832] request split points lookup for chunk offerStore.offer { : 1745250112 } -->> { : 1745287547 }
Mon Oct 21 16:16:27.559 [conn2428949] end connection 172.16.64.140:55687 (6256 connections now open)
Mon Oct 21 16:16:27.561 [initandlisten] connection accepted from 172.16.64.140:55689 #2428959 (6258 connections now open)
Mon Oct 21 16:16:27.587 [conn2428959]  authenticate db: local { authenticate: 1, nonce: "ba3cd2e155d82a07", user: "__system", key: "53846d3825d70242ef7932055a158e96" }
Mon Oct 21 16:16:29.375 [conn2380930] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 276924 } cursorid:3770777652693921886 ntoreturn:0 keyUpdates:0 numYields: 262 locks(micros) r:1370535 nreturned:1093 reslen:122820 3453ms
Mon Oct 21 16:16:29.474 [conn2418832] request split points lookup for chunk offerStore.offer { : 905389447 } -->> { : 905603317 }
Mon Oct 21 16:16:30.435 [conn2425969] query offerStore.offer query: { exportFP.MAPPED: { $exists: 1 }, shopId: 216106 } ntoreturn:1 ntoskip:0 nscanned:7576 keyUpdates:0 numYields: 124 locks(micros) r:3174937 nreturned:0 reslen:20 3487ms
Mon Oct 21 16:16:30.732 [conn2425600] query offerStore.offer query: { exportFP.MAPPED: { $exists: 1 }, shopId: 215890 } ntoreturn:1 ntoskip:0 nscanned:24308 keyUpdates:0 numYields: 143 locks(micros) r:1433707 nreturned:0 reslen:20 3948ms
Mon Oct 21 16:16:30.881 [conn2426019] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 279648 } ntoreturn:1 ntoskip:0 nscanned:20752 keyUpdates:0 numYields: 139 locks(micros) r:1408176 nreturned:0 reslen:20 3882ms
Mon Oct 21 16:16:31.878 [conn2426338] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 199857 } ntoreturn:1 ntoskip:0 nscanned:36044 keyUpdates:0 numYields: 242 locks(micros) r:3262694 nreturned:0 reslen:20 5241ms
Mon Oct 21 16:16:32.773 [conn2427536] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 282123 } ntoreturn:1 ntoskip:0 nscanned:59376 keyUpdates:0 numYields: 336 locks(micros) r:3164849 nreturned:0 reslen:20 6079ms
Mon Oct 21 16:16:32.946 [conn2417498] query offerStore.offer query: { exportFP.MAPPED: { $exists: 1 }, shopId: 278643 } ntoreturn:1 ntoskip:0 nscanned:66302 keyUpdates:0 numYields: 366 locks(micros) r:3208253 nreturned:0 reslen:20 6275ms
Mon Oct 21 16:16:33.034 [conn2423845] request split points lookup for chunk offerStore.offer { : 588187276 } -->> { : 589508184 }
Mon Oct 21 16:16:33.324 [conn2420761] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 21119 } cursorid:3771175798665721828 ntoreturn:0 keyUpdates:0 numYields: 6373 locks(micros) r:30854145 nreturned:41331 reslen:4194400 101105ms
Mon Oct 21 16:16:33.557 [conn2424118] request split points lookup for chunk offerStore.offer { : 1739626209 } -->> { : 1739667648 }
Mon Oct 21 16:16:33.907 [conn2425606] query offerStore.offer query: { exportFP.ANP: { $exists: 1 }, shopId: 279864 } ntoreturn:1 ntoskip:0 nscanned:77150 keyUpdates:0 numYields: 446 locks(micros) r:3538169 nreturned:0 reslen:20 5751ms
Mon Oct 21 16:16:35.168 [conn2422966] end connection 172.16.64.143:52128 (6256 connections now open)
Mon Oct 21 16:16:37.016 [conn2428653] end connection 172.16.64.143:52443 (6255 connections now open)
Mon Oct 21 16:16:37.021 [conn2413957] getmore offerStore.offer query: { exportFP: { $exists: 1 }, shopId: 281116 } cursorid:3771186280175124980 ntoreturn:0 keyUpdates:0 numYields: 7080 locks(micros) r:31094757 nreturned:31071 reslen:3483153 104202ms

depends on

SERVER-9622 establishing cursor for query can fail with stale config after first batch built - testshard1.js failing on Windows

Closed

duplicates

SERVER-5752 query yielding with frequent migrations can cause repeated stale config

Closed

Assignee:: Kelsey Schubert

Reporter:: Kay Agahd

Participants:: Kay Agahd, Kelsey Schubert

Votes:: 3 Vote for this issue

Watchers:: 11 Start watching this issue

Created:: Oct 22 2013 01:15:14 PM UTC

Updated:: Dec 07 2016 10:19:53 PM UTC

Resolved:: Sep 06 2016 06:35:47 PM UTC

Details

Description

Attachments

Issue Links

Activity

People

Dates