Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-165

Support new updateDescription field truncatedArrays and dot format for updatedFields

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • 1.14.0
    • Affects Version/s: None
    • Component/s: Source
    • None

      New fields have been added to updatedDescription:

      updateDescription: {
        updatedFields: {},
        truncatedArrays: [{field: "arrayField", newSize: 2}]  // New field.
        disambiguatedPaths: { "residences.0.0": [ "residences", 0, "0" ] } // New since MongoDB 6.0
      }
      

      Along with the truncatedArrays field comes partial updates in the updatedFields document.

      The original complexity is this add ambiguity to the updateFields paths where an update to "a.b.0.c.1.d" could update either:

      { a:  {b: [{c: [<val0>, {d: "field_to_be_updated"}, …]}, …]} }
      
      // Or
      { "a.b": {"0": {"c.1": {d: "field_to_be_updated"}}}
      

      The syntax required for the update is different and currently not supported. As such, the existing code checks for unsupported updateDescription fields and throws an unsupported exception - users are required to use fullDocument

      However, Since Mongo 6.0 expanded events have been supported. This adds disambiguatedPaths to the Update Event.

      Adding a new configuration to the connector could allow for supporting expanded events, this would allow the connector to support both truncatedArrays and partial updates.

            Assignee:
            Unassigned Unassigned
            Reporter:
            ross@mongodb.com Ross Lawley
            Votes:
            5 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated: