For users not using case classes or java beans declaring the schema for unsupported bson types and querying them is painful.

Declaring schema for unsupported bson types requires indepth knowledge of the best StructType add helpers so users can easily declare the schema. Querying those unsupported bson types also requires the same indepth knowledge of the StructType add helpers to make this process easier.

Was:

Using .filter on top of spark connector is dropping rows from dataframe

I'm Pulling data from Mongo Altas into a spark dataframe using Spark-connector 1.0.0 with Python. I'm specifying my schema explicitly due to combined datatypes in some fields.

When I specify a filter (.filter, .where, sql query behave identically here) before evaluating, AND that filter is for a string comparison, AND I specify equivalency (==), I always get zero results. Some troubleshooting:
--Evaluating without filters returns all data as it should.
--Evaluating the same field with a different boolean expression (!=) works as it should.
--Evaluating using the same expression on local data works fine. This holds for local dummy data, as well as cached results. My current workaround is to cache all of the data prior to applying filters.