-
Type: Bug
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.4.5
-
Component/s: mongoimport
-
None
Updated description:
mongoimport appears to be able to send BSON documents greater than 16MB to the server, causing a server-side assertion. Instead, it should detect too-large documents and give a useful error message that helps users identify the problem document. Whether the document is skipped or the import halts should continue to be controlled by --stopOnError.
Original description:
When importing the attached document using mongoimport, the import fails and the server logs a backtrace. I tried numerous options including batchSize and it does not seem to work. None of the individual documents are over the 16MB limit.
Below are my commands:
mongoimportdev@dev20:~$ mongoimport --version mongoimport version: r3.4.5 git version: 520b8f3092c48d934f0cd78ab5f40fe594f96863 Go version: go1.7 os: linux arch: amd64 compiler: gc OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016 dev@dev20:~$ mongoimport --db borg --collection packetlog --batchSize=1 --stopOnError -vv --drop --file requests.json 2017-06-20T21:13:39.189+0200 using 0 decoding workers 2017-06-20T21:13:39.189+0200 using 1 insert workers 2017-06-20T21:13:39.189+0200 filesize: 50414859 bytes 2017-06-20T21:13:39.189+0200 using fields: 2017-06-20T21:13:39.190+0200 will listen for SIGTERM, SIGINT, and SIGKILL 2017-06-20T21:13:39.191+0200 connected to: localhost 2017-06-20T21:13:39.191+0200 ns: borg.packetlog 2017-06-20T21:13:39.191+0200 connected to node type: standalone 2017-06-20T21:13:39.191+0200 standalone server: setting write concern w to 1 2017-06-20T21:13:39.191+0200 using write concern: w='1', j=false, fsync=false, wtimeout=0 2017-06-20T21:13:39.191+0200 dropping: borg.packetlog 2017-06-20T21:13:39.192+0200 standalone server: setting write concern w to 1 2017-06-20T21:13:39.194+0200 using write concern: w='1', j=false, fsync=false, wtimeout=0 2017-06-20T21:13:40.196+0200 Failed: lost connection to server 2017-06-20T21:13:40.196+0200 imported 188 documentsThe backtrace is below:
mongo log file2017-06-20T21:13:40.186+0200 I - [conn95] Assertion: 10334:BSONObj size: 19805143 (0x12E33D7) is invalid. Size must be between 0 and 16793600(16MB) First element: insert: "packetlog" src/mongo/bson/bsonobj.cpp 58 2017-06-20T21:13:40.195+0200 I CONTROL [conn95] 0x5591527debe1 0x559152769a37 0x559151a79905 0x559151a799e4 0x559151b137b8 0x559151dc2d60 0x559151ee73f9 0x559151ee94c6 0x559151ae842d 0x559151ae8d6d 0x559152744c52 0x7fe2c69926da 0x7fe2c66cc17f ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"559151268000","o":"1576BE1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"559151268000","o":"1501A37","s":"_ZN5mongo10logContextEPKc"},{"b":"559151268000","o":"811905","s":"_ZN5mongo23msgassertedWithLocationEiPKcS1_j"},{"b":"559151268000","o":"8119E4","s":"_ZN5mongo30msgassertedNoTraceWithLocationEiPKcS1_j"},{"b":"559151268000","o":"8AB7B8","s":"_ZNK5mongo7BSONObj14_assertInvalidEv"},{"b":"559151268000","o":"B5AD60","s":"_ZN5mongo9DbMessage9nextJsObjEv"},{"b":"559151268000","o":"C7F3F9"},{"b":"559151268000","o":"C814C6","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"},{"b":"559151268000","o":"88042D","s":"_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE"},{"b":"559151268000","o":"880D6D"},{"b":"559151268000","o":"14DCC52"},{"b":"7FE2C698B000","o":"76DA"},{"b":"7FE2C65C4000","o":"10817F","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.4.4", "gitVersion" : "888390515874a9debd1b6c5d36559ca86b44babd", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.10.0-21-generic", "version" : "#23-Ubuntu SMP Fri Apr 28 16:14:22 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "559151268000", "elfType" : 3, "buildId" : "5754B63E6FC9683BABEC9CDF7BF6764E6939C8A2" }, { "b" : "7FFEC59C0000", "elfType" : 3, "buildId" : "6AD662AE66D2690703876837CAF3524BE77CE78C" }, { "b" : "7FE2C7919000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "4957783883176B8025AF45D8E771B92C46A85A84" }, { "b" : "7FE2C74D5000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "446FE5C583238974E0BD57B54D589591854F664C" }, { "b" : "7FE2C72CD000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "375C37AE53F311E5F68DAD4890D953E88FEB226B" }, { "b" : "7FE2C70C9000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "A021F9F70C343C23E2DEB7DAEA4C1A3FC053DBC7" }, { "b" : "7FE2C6DC0000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "4A736E6C4EFF6FD9BFC51A281F9EABBC7C9CA707" }, { "b" : "7FE2C6BA9000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "BD7D7479293DD928CA0A4769701EF66A4B0D0D29" }, { "b" : "7FE2C698B000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "236FBBFD91B7DC1EDE76916401A1435A1DA42114" }, { "b" : "7FE2C65C4000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "5B72576FF331E93852355123AFECDEC70FD247B5" }, { "b" : "7FE2C7B82000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9BD92DC65FF841703504FBB338CA807C1F15A25C" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x5591527debe1] mongod(_ZN5mongo10logContextEPKc+0x177) [0x559152769a37] mongod(_ZN5mongo23msgassertedWithLocationEiPKcS1_j+0x1C9) [0x559151a79905] mongod(_ZN5mongo30msgassertedNoTraceWithLocationEiPKcS1_j+0x0) [0x559151a799e4] mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x208) [0x559151b137b8] mongod(_ZN5mongo9DbMessage9nextJsObjEv+0x110) [0x559151dc2d60] mongod(+0xC7F3F9) [0x559151ee73f9] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x6D6) [0x559151ee94c6] mongod(_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE+0x1ED) [0x559151ae842d] mongod(+0x880D6D) [0x559151ae8d6d] mongod(+0x14DCC52) [0x559152744c52] libpthread.so.0(+0x76DA) [0x7fe2c69926da] libc.so.6(clone+0x5F) [0x7fe2c66cc17f] ----- END BACKTRACE ----- 2017-06-20T21:13:40.195+0200 I - [conn95] AssertionException handling request, closing client connection: 10334 BSONObj size: 19805143 (0x12E33D7) is invalid. Size must be between 0 and 16793600(16MB) First element: insert: "packetlog"
- duplicates
-
TOOLS-1550 Prevent insertion of overlarge documents
- Closed