-
Type: Improvement
-
Resolution: Fixed
-
Priority: Minor - P4
-
Affects Version/s: 2.3.0
-
Component/s: Partitioners
-
None
Since fields that explicitly aren't nullable are added to the filters (for pruning), the MongoSamplePartitioner needs to perform a full scan to count the filtered rows.
In some cases, it is very slow, especially on large data set. I notice that mongo-connector 2.0.0 uses an inaccurate count instead of performing a count, so that the problem does not exists.
Is it possible to add some warning information or document somewhere to prevent someone from misusing the InNullable schema qualifier.