-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
Affects Version/s: 3.2.5, 3.2.7
-
Component/s: WiredTiger
-
None
-
Fully Compatible
-
ALL
-
v3.2
Test does some inserts to generate checkpoints of about 30 seconds, while doing createIndex about once per second:
The createIndexes stall during checkpoints, which in turn stalls other operations because createIndex holds some locks.
Stack traces collected during the stall show createIndex stuck waiting on a mutex in __wt_curfile_open:
Thread 3 (Thread 0x7fe1682e1700 (LWP 22695)): #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007fe16e399cfd in __GI___pthread_mutex_lock (mutex=0x32d6440) at ../nptl/pthread_mutex_lock.c:80 #2 0x00000000019fe649 in __wt_curfile_open () #3 0x0000000001a63a05 in __session_open_cursor_int () #4 0x0000000001a14d33 in __wt_curtable_open () #5 0x0000000001a63889 in __session_open_cursor_int () #6 0x0000000001a63e19 in __session_open_cursor () #7 0x0000000001066119 in mongo::WiredTigerIndex::BulkBuilder::openBulkCursor(mongo::WiredTigerIndex*) () #8 0x000000000106137e in mongo::WiredTigerIndexStandard::getBulkBuilder(mongo::OperationContext*, bool) () #9 0x0000000000c79fa9 in mongo::IndexAccessMethod::commitBulk(mongo::OperationContext*, std::unique_ptr<mongo::IndexAccessMethod::BulkBuilder, std::default_delete<mongo::IndexAccessMethod::BulkBuilder> >, bool, bool, std::set<mongo::RecordId, std::less<mongo::RecordId>, std::allocator<mongo::RecordId> >*) () #10 0x0000000000ac29b0 in mongo::MultiIndexBlock::doneInserting(std::set<mongo::RecordId, std::less<mongo::RecordId>, std::allocator<mongo::RecordId> >*) () #11 0x0000000000ac2f00 in mongo::MultiIndexBlock::insertAllDocumentsInCollection(std::set<mongo::RecordId, std::less<mongo::RecordId>, std::allocator<mongo::RecordId> >*) () #12 0x0000000000aedf86 in mongo::CmdCreateIndex::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&) () #13 0x0000000000b816e3 in mongo::Command::run(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) () #14 0x0000000000b82554 in mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) () #15 0x0000000000ad98c0 in mongo::runCommands(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) () #16 0x0000000000c9b095 in mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) () #17 0x0000000000961c2c in mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*) () #18 0x00000000012b508d in mongo::PortMessageServer::handleIncomingMsg(void*) () #19 0x00007fe16e3976aa in start_thread (arg=0x7fe1682e1700) at pthread_create.c:333 #20 0x00007fe16e0cd13d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
Here's the repro code:
function insert(threads) {
x = ''
for (var i=0; i<1000; i++)
x += 'x'
docs = []
for (var i=0; i<1000; i++)
docs.push({x:x})
ops = [{
op: "insert",
ns: "test.c",
doc: docs,
}]
res = benchRun({
ops: ops,
seconds: 10000,
parallel: threads
})
printjson(res)
}
function create() {
for (var i=0; i<150; i++) {
c = db['c'+i]
t0 = new Date()
print('createIndex at', t0)
db['c'+i].createIndex({x:1})
print('took', new Date() - t0, 'ms')
sleep(1000)
}
}
- related to
-
WT-2781 Enhance bulk cursor option with an option to return immediately on contention
-
- Closed
-