Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-75

New id strategy that doesn't reuse configuration parameters.

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 1.2.0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Needed
    • Hide

      `PartialValueStrategy` and `PartialKeyStrategy` now have namespaced configurations. So use:

      `document.id.strategy.partial.value.projection.type`, `document.id.strategy.partial.value.projection.list`,
      `document.id.strategy.partial.key.projection.type` and `document.id.strategy.partial.key.projection.list`.

      instead of [key|value].projection.[type|list].

      Remove note about PROJECTION POST PROCESSORS ARE NOT COMPATIBLE WITH PARTIALVALUESTRATEGY

      Show
      `PartialValueStrategy` and `PartialKeyStrategy` now have namespaced configurations. So use: `document.id.strategy.partial.value.projection.type`, `document.id.strategy.partial.value.projection.list`, `document.id.strategy.partial.key.projection.type` and `document.id.strategy.partial.key.projection.list`. instead of [key|value] .projection. [type|list] . Remove note about PROJECTION POST PROCESSORS ARE NOT COMPATIBLE WITH PARTIALVALUESTRATEGY

      We want to support a solution to config more than one processor on the chain of post processors.

      Consider the following example:

        "topic.override.source.document.id.strategy":"com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy",
     "topic.override.source.collection":"sink",
     "topic.override.source.value.projection.type":"whitelist",
     "topic.override.source.value.projection.list":"attuid",
     "topic.override.source.writemodel.strategy": "com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy",

     "topic.override.source.post.processor.chain":"com.mongodb.kafka.connect.sink.processor.WhitelistValueProjector",
     "topic.override.source.collection":"sink",
     "topic.override.source.value.projection.type":"whitelist",
     "topic.override.source.value.projection.list":"attuid, name, pc",
     "topic.override.source.batch.size":"100","name":"mongo-sink"}

      Currently, this fails to update correctly the MongoDB documents. The current design only allows one processor to be configured as it is not possible to configure settings for each of the processors.

      This work is to implement a new id strategy that does not reuse configuration parameters.

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            seth.payne@mongodb.com Seth Payne
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: