Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-88314

[Streams] Support large single group accumulator checkpoint

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Atlas Streams

      The existing checkpoint code doesn't accommodate single large accumulator state larger than 128MB. In Atlas stream processing, the maximum size for a single large accumulator stat is 1GB. We need to add support for checkpointing these large states. 

      https://github.com/10gen/mongo/blob/a01f7c5fd62674c1bbc5531ae35f23b007690cad/src/mongo/db/modules/enterprise/src/streams/exec/checkpoint/local_disk_checkpoint_storage.cpp#L118

      https://github.com/10gen/mongo/blob/a01f7c5fd62674c1bbc5531ae35f23b007690cad/src/mongo/db/modules/enterprise/src/streams/exec/checkpoint/local_disk_checkpoint_storage.h#L32

            Assignee:
            mayuresh.kulkarni@mongodb.com Mayuresh Kulkarni
            Reporter:
            harendra.chawla@mongodb.com Harendra Chawla
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: