Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-224

Create new Sink Connector for bucket pattern

    • Type: Icon: New Feature New Feature
    • Resolution: Duplicate
    • Priority: Icon: Unknown Unknown
    • None
    • Affects Version/s: None
    • Component/s: Sink
    • Environment:
      All
    • Needed
    • Hide

      The usage and settings for the UpdateOneBusinessKeyBucketStrategy are listed below.

      Show
      The usage and settings for the UpdateOneBusinessKeyBucketStrategy are listed below.

      Will take a document or an array of documents from a path in the source and bucket them to an array field in the destination document. In addition will also add an _insertedTS (inserted timestamp) and _modifiedTS (modified timestamp) fields into documents.

      Set the following configuration: document.id.strategy=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy
      document.id.strategy.partial.value.projection.list=businessKeyField
      document.id.strategy.partial.value.projection.type=AllowList

      writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneBusinessKeyBucketStrategy
      writemodel.strategy.bucketing.source.field=dataPayLoadField
      writemodel.strategy.bucketing.source.field.multiple=false
      writemodel.strategy.bucketing.target.array=bucketField
      writemodel.strategy.bucketing.target.array.size=3
      writemodel.strategy.bucketing.target.array.unique=true

        1. MongoSinkTopicConfig.java
          45 kB
          Naresh Maharaj
        2. UpdateOneBusinessKeyBucketStrategy.java
          9 kB
          Naresh Maharaj

            Assignee:
            robert.walters@mongodb.com Robert Walters (Inactive)
            Reporter:
            naresh.maharaj@mongodb.com Naresh Maharaj (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: