To test the mongoDb sharding capabilities we have continuously inserted data into a two shards environment. Each shard consists of
one replica set (1 Master, 1 Secondary and 1 Arbiter). At a given point we perform some find operations to verify whether all data is returned. The problem is that there are sometimes gaps in the result. E.g. the result contains document 5200 and 5202 but not document 5201. We are using the mongo java driver 2.6.2 but could also experience the behavior with the php driver.

These gaps are independent of the used write concern.

To reproduce the problem a java maven project is attached to this issue:

• Follow the steps in the README.txt to run the test

Details:

• The test application provides 3 classes with a main method located in src\main\java\com\seitenbau\testing\mongo\runner: Inserter, VerifierSharding, VerifierFinding
• The Inserter inserts documents that contain a field storing a counter (the counter is incremented each time a document is inserted). The script generates the file "counter-state" which stores the current value of the counter.
• The VerifierSharding logs some information on sharding and counting (verifier-sharding-info.log).
• The VerifierFinding performs a find operation to find all documents that are saved in the target collection at the moment.
  The result set is sorted and then logged in a file with the format <timestamp>-node-data. Additionally the script generates the log file "verifier-finding-info.log".
• To configure the application check the "config.properties" file
• You have to abort the scripts manually as they perform their intended job inside a while loop.