Schema inference for documents nested in arrays falls back to "string" when any difference is detected in the schemas for the nested documents. This is necessary because Kafka schemas can not handle arrays with elements of different type. But we can improve the schema inference to detect some cases where the schemas for the nested documents are actually compatible:
- Where the field is present in one document but missing in another
- Where the field is present in one document but null in another
- Where the field types conflict (in this case we can push the conflict down to the schema for the field)
- Where the field is an array with elements of some type in one document but an empty array in another