-
Type: Bug
-
Resolution: Unresolved
-
Priority: Minor - P4
-
None
-
Affects Version/s: None
-
Component/s: Node.js
How frequently does the bug occur?
Always
Description
We have some VERY long running "write" queries, but to be fair, against large datasets. Although I don't really know what "large" means in Realm terms... I am talking when we have ~1,000,000 items to write/create (see code below). It can take upwards of 4 minutes (depending on the device). And because it is wrapped in a single write statement, our main thread in our Node/Electron app is blocked during this time (which is the real problem). I would be okay with it taking 4 minutes but cannot have our main thread blocked.
Here are my questions:
1) is this expected that it could be taking 4 minutes to make this large Write query? or is there something wrong with the way we structure our code below?
2) we wrap it in a single Write statement because that is what is recommended from the docs, but does that change when you have such a large dataset being written like this? Should we instead be doing some sort of batching of the writes?
3) is there any way to run Realm write queries asynchronously in a way that would not be blocking the main thread? I know in the past the Realm team has said there is not a way to run Realm in a Node Worker Thread, is this still true? (see here: https://github.com/realm/realm-js/issues/4434#issuecomment-1075403729 and here: https://github.com/realm/realm-js/issues/4440)
The TLDR here is whether this is expected to take a long time for a creation of 1 million items and whether there is a better way to structure our write (should we batch it breaking it up into 10 x 100,000 writes for example?) and/or if there is a way to not block the main thread in Node by having Realm run asynchronously and/or off thread in like a Worker Thread?
Thanks so much in advance!
Stacktrace & log output
N/A
Can you reproduce the bug?
Always
Reproduction Steps
Here is the example code. For the context of this issue we are having, between the "parent" RootBlocks and their children, they total around 1.1 million RootBlocks and Blocks being created.
try { client.write(() => { blocksMap.forEach((blockObj, blockId) => { //Create Root block let _id = new BSON.ObjectId(); let rootTitle = blockObj.block?.title; const parentRootBlock = client.create('RootBlock', { _id: _id, string: rootTitle, children: [], }); //Create direct children of root block blockObj.children.forEach(childBlock => { let _id = new BSON.ObjectId(); const blockContent = blocksMap.get(childBlock)?.block?.string; const eachChild = client.create('Block', { _id: _id, string: blockContent, children: [], root: parentRootBlock, }); //Add the child to the root block children parentRootBlock.children.push(eachChild); }); }); }); } catch (error) { console.log(`RealmDB writeToRealm error: ${error}`); } finally { }
Version
10.22
What services are you using?
Local Database only
Are you using encryption?
No
Platform OS and version(s)
Windows 11
Build environment
N/A
Cocoapods version
No response