Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-86746

kafka consumer group doesn't stay active

    • Atlas Streams
    • Fully Compatible
    • Sprint 43, Sprint 44

      Joe created a stream processor with consumer group ID. It is successfully committing offsets to the consumer group, but it does not show up in the Confluent UI.

      It's because the stream processor is not reporting itself as an active member of the consumer group. This breaks important kafka tooling. 

      (python3-venv) ubuntu@ip-10-122-3-49:~/binaries/kafka/kafka_2.12-3.6.1/bin$ ./kafka-consumer-groups.sh --bootstrap-server "pkc-4r087.us-west2.gcp.confluent.cloud:9092" --command-config ~/.confluent_kafka --describe --group test1
      
      Consumer group 'test1' has no active members.
      
      GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
      test1           topic_0         4          698             698             0               -               -               -
      test1           topic_0         5          640             640             0               -               -               -
      test1           topic_0         2          691             691             0               -               -               -
      test1           topic_0         3          650             650             0               -               -               -
      test1           topic_0         1          718             718             0               -               -               -
      test1           topic_0         0          635             635             0               -               -               - 

       

      It is because we are not keeping the consumer group active (we only make it active when we update the committed offsets. for some reason we decided to actually read using another client).

       

            Assignee:
            matthew.normyle@mongodb.com Matthew Normyle
            Reporter:
            matthew.normyle@mongodb.com Matthew Normyle
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: