Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-128

Mongo source connector offset partition map

    • Type: Icon: New Feature New Feature
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 1.3.0
    • Affects Version/s: None
    • Component/s: Source
    • None

      Currently offset creates as a concatenation of connection.uri, database and collection parameters.
      https://github.com/mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/source/MongoSourceTask.java#L309-L317

      This approach has a couple of disadvantages.

      1) such offset could contain mongo username and password, which will be written to kafka connect offsets topic. I guess it's not very secure.
      2) if the mongo password or username was changed, the offset will be lost.

      Is it make sense to use a static partition name? Or in the future, there is could be more than one partition?

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            andreworty@gmail.com Andrey B
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: