Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-764

mongorestore v 3.x fails with invalid BSON size exception

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.3, 3.1.3
    • Component/s: None
    • None

      While trying to reply the oplog using mongorestore from MongoDB 3.x on OSX, i.e., mongorestore --oplogReply, I encountered a failure:

      $ mongorestore --oplogReplay
      2015-05-29T12:10:43.515+0100	using default 'dump' directory
      2015-05-29T12:10:43.528+0100	building a list of dbs and collections to restore from dump dir
      2015-05-29T12:10:43.529+0100	replaying oplog
      2015-05-29T12:10:45.111+0100	Failed: restore error: error applying oplog: applyOps: EOF
      

      The mongod log file shows two stack traces and errors relating to invalid BSONObj size (running mongod server and mongorestore from MongoDB 3.0.3):

      2015-05-29T12:10:45.102+0100 I -        [conn3] Assertion: 10334:BSONObj size: 18085059 (0x113F4C3) is invalid. Size must be between 0 and 16793600(16MB) First elem
      ent: 0: { ts: Timestamp 1431553500000|1, unsigned char: 8874020502223060881, void: 2, op: "u", ns: "arrow.client2", unsigned __int128: { $set: { lastConnectionDate:
       new Date(1431553500003) } }, o2: { _id: "2G24KEXKyCdaFrSH" } }
      ...
       mongod(mongo::printStackTrace(std::__1::basic_ostream<char, std::__1::char_traits<char> >&)+0x39) [0x10634c849]
       mongod(mongo::logContext(char const*)+0x100) [0x1062fed10]
       mongod(mongo::msgasserted(int, char const*)+0x13A) [0x1062ece8a]
       mongod(mongo::msgasserted(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)+0x1A) [0x1062ecd4a]
       mongod(mongo::BSONObj::_assertInvalid() const+0x4CE) [0x105bfda7e]
       mongod(mongo::BSONElement::embeddedObject() const+0x6A) [0x105bf646a]
       mongod(mongo::BSONElement::toString(mongo::StringBuilderImpl<mongo::TrivialAllocator>&, bool, bool, int) const+0x387) [0x105bf9927]
       mongod(mongo::BSONElement::toString(bool, bool) const+0x58) [0x105bf6888]
       mongod(mongo::BSONObj::_assertInvalid() const+0x414) [0x105bfd9c4]
       mongod(mongo::DbMessage::nextJsObj()+0xA0) [0x105dd8c80]
       mongod(mongo::QueryMessage::QueryMessage(mongo::DbMessage&)+0x5D) [0x105c3e1ad]
       mongod(mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&)+0x3F4) [0x105e9e414]
       mongod(mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*, mongo::LastError*)+0xDE) [0x105be1e6e]
       mongod(mongo::PortMessageServer::handleIncomingMsg(void*)+0x33C) [0x10630e8ac]
       mongod(boost::(anonymous namespace)::thread_proxy(void*)+0xB1) [0x106380271]
       libsystem_pthread.dylib(_pthread_body+0x83) [0x7fff926e5268]
       libsystem_pthread.dylib(_pthread_body+0x0) [0x7fff926e51e5]
       libsystem_pthread.dylib(thread_start+0xD) [0x7fff926e341d]
      -----  END BACKTRACE  -----
      2015-05-29T12:10:45.108+0100 I -        [conn3] Assertion: 10334:BSONObj size: 18085074 (0x113F4D2) is invalid. Size must be between 0 and 16793600(16MB) First element:
      ...
       mongod(mongo::printStackTrace(std::__1::basic_ostream<char, std::__1::char_traits<char> >&)+0x39) [0x10634c849]
       mongod(mongo::logContext(char const*)+0x100) [0x1062fed10]
       mongod(mongo::msgasserted(int, char const*)+0x13A) [0x1062ece8a]
       mongod(mongo::msgasserted(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)+0x1A) [0x1062ecd4a]
       mongod(mongo::BSONObj::_assertInvalid() const+0x4CE) [0x105bfda7e]
       mongod(mongo::DbMessage::nextJsObj()+0xA0) [0x105dd8c80]
       mongod(mongo::QueryMessage::QueryMessage(mongo::DbMessage&)+0x5D) [0x105c3e1ad]
       mongod(mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&)+0x3F4) [0x105e9e414]
       mongod(mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*, mongo::LastError*)+0xDE) [0x105be1e6e]
       mongod(mongo::PortMessageServer::handleIncomingMsg(void*)+0x33C) [0x10630e8ac]
       mongod(boost::(anonymous namespace)::thread_proxy(void*)+0xB1) [0x106380271]
       libsystem_pthread.dylib(_pthread_body+0x83) [0x7fff926e5268]
       libsystem_pthread.dylib(_pthread_body+0x0) [0x7fff926e51e5]
       libsystem_pthread.dylib(thread_start+0xD) [0x7fff926e341d]
      -----  END BACKTRACE  -----
      2015-05-29T12:10:45.111+0100 I NETWORK  [conn3] AssertionException handling request, closing client connection: 10334 BSONObj size: 18085074 (0x113F4D2) is invalid. Size must be between 0 and 16793600(16MB) First element:
      

      Running the same command from the MongoDB 2.6.9 release it works.

      I've tested this with different combinations:

      • 3.x mongorestore vs 3.x mongod - failure (BSONObj size assertion)
      • 3.x mongorestore vs 2.6.9 mongod - failure (BSONObj size assertion)
      • 2.6.9 mongorestore vs 3.x mongod - failure (expected - due to auth schema version differences)
      • 2.6.9 mongorestore vs 2.6.9 mongod - success

      Based on this evidence it looks like the problem is with mongorestore in 3.x

            Assignee:
            kyle.erf Kyle Erf
            Reporter:
            ronan.bohan@mongodb.com Ronan Bohan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: