Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-6844

Invalid BSONObj size error causing secondary optime freeze

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.1.0
    • Component/s: Replication
    • None
    • Environment:
      Azure Worker Role
    • ALL

      One of our production secondary databases now has an optime that is nearly 24 hours old. Looking at the logs at that optime I see some assertions

      [rsSync] Assertion: 10334:Invalid BSONObj size: 1818322244 (0x4465616C) first element: r: ?type=101; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] replSet syncTail: 10334 Invalid BSONObj size: 1818322244 (0x4465616C) first element: r: ?type=101, syncing: { ts: Timestamp 1345750316000|66, h: 1018104994802725908, op: ""u"", ns: ""xq.SessionModel"", o2:

      { _id: ""fOLao2S5-cu8sYYVWuAmOr_t77XseH8rN0t7nt-fNUI"" }

      , o:

      { _id: ""fOLao2S5-cu8sYYVWuAmOr_t77XseH8rN0t7nt-fNUI"", Expires: new Date(1345752117253), Id: ""damcomm"", AccountId: BinData, Roles: [ ""Dealer"" ], AuthApiToken: null }

      }; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] replSet not trying to sync from 10.26.10.41:27017, it is vetoed for 269 more seconds; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] replSet syncing to: 10.26.2.151:27017; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] Assertion: 10334:Invalid BSONObj size: 1818322244 (0x4465616C) first element: r: ?type=101; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] replSet syncTail: 10334 Invalid BSONObj size: 1818322244 (0x4465616C) first element: r: ?type=101, syncing: { ts: Timestamp 1345750316000|66, h: 1018104994802725908, op: ""u"", ns: ""xq.SessionModel"", o2:

      { _id: ""fOLao2S5-cu8sYYVWuAmOr_t77XseH8rN0t7nt-fNUI"" }

      , o:

      { _id: ""fOLao2S5-cu8sYYVWuAmOr_t77XseH8rN0t7nt-fNUI"", Expires: new Date(1345752117253), Id: ""damcomm"", AccountId: BinData, Roles: [ ""Dealer"" ], AuthApiToken: null }

      }; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] replSet not trying to sync from 10.26.10.41:27017, it is vetoed for 238 more seconds; TraceSource 'WaWorkerHost.exe' event"
      [rsSync] replSet not trying to sync from 10.26.2.151:27017, it is vetoed for 269 more seconds; TraceSource 'WaWorkerHost.exe' event"

        1. Secondary.log
          128 kB
          John Woakes

            Assignee:
            benjamin.becker Ben Becker
            Reporter:
            woaksie John Woakes
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: