-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Blocker - P1
-
None
-
Affects Version/s: None
-
Component/s: None
Goals
Over at WWDC.io we are experiencing a crash in Realm v5.2/5.3 that occurs shortly after application launch. The crash shows symptoms you might expect from a use-after-free memory situation.
As part of upgrading to Realm 5, we added support for compacting the Realm based on a free space threshold. After #6629 was addressed, we started running manual tests around upgrading from the current version of WWDC.io to the version containing Realm 5.
I noticed it always crashed on the first launch of the app with Realm 5 but never on subsequent launches. The stack trace where the EXC_BAD_ACCESS was encountered never seemed to be the same. I looked through several recent issues reported on GitHub for crashes related to Realm 5 for similar symptoms. Of note are #6628 and #6555 . Two of our team members have seen crashes with stack traces that end in realm::util::EncryptedFileMapping::read_barrier which is how those 2 issues might be related to this issue.
But like I said, the exact stack trace changes from run to run. I think it is because the crash is occurring upon the first commit after a Realm has been compacted. Or, so it seems from examining all the threads in the application.
I also see the following on our background Realm queue:
#13 0x0000000102a5fab3 in realm::_impl::RealmCoordinator::commit_write(realm::Realm&) at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/realm_coordinator.cpp:688 #14 0x0000000102b3800a in realm::Realm::commit_transaction() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/shared_realm.cpp:671 #15 0x0000000102b03d9f in -[RLMRealm commitWriteTransactionWithoutNotifying:error:] at /Users/realm/workspace/cocoa-pipeline/Realm/RLMRealm.mm:698 #16 0x000000010275aebf in Realm.commitWrite(withoutNotifying:) [inlined] at /Users/realm/workspace/cocoa-pipeline/RealmSwift/Realm.swift:291 #17 0x000000010275ae79 in Realm.write<A>(withoutNotifying:_:) at /Users/realm/workspace/cocoa-pipeline/RealmSwift/Realm.swift:232 #18 0x0000000101ab0a4e in closure #2 in Storage.modify<A>(_:with:) at /Users/allen.humphreys/personal_projects/WWDC/ConfCore/Storage.swift:489
And in one of the notification listener threads we see it processing notifications:
#9 0x0000000102a7445f in realm::_impl::ResultsNotifier::run() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/results_notifier.cpp:158
After scratching my head for quite some time and poking at different pieces of our code, I disabled the compact on launch and found that the crash did not occur. So I concluded that the issue likely lies in Realm and I've opened this ticket in the hopes that we can get to the bottom of it!
Expected Results
We'd like to not crash (and to support compacting the realm).
Actual Results
The app crashes. The stack trace is different from run to run, which is part of why I suggested it's a memory-related bug.
<details>
<summary>A Stack Trace</summary>
Thread 1 Queue : com.apple.main-thread (serial) #0 0x0000000102bfa9d7 in realm::Array::init_from_mem(realm::MemRef) () #1 0x0000000102a460b7 in realm::Array::init_from_ref(unsigned long) at /Users/realm/workspace/cocoa-pipeline/core/include/realm/array.hpp:169 #2 0x0000000102c9ad10 in realm::Cluster::init_leaf(realm::ColKey, realm::ArrayPayload*) const () #3 0x0000000102abf097 in realm::LinkMap::set_cluster(realm::Cluster const*) at /Users/realm/workspace/cocoa-pipeline/core/include/realm/query_expression.hpp:1863 #4 0x0000000102b7ea3e in realm::ParentNode::set_cluster(realm::Cluster const*) () #5 0x0000000102b7f09a in realm::util::FunctionRef<bool (realm::Cluster const*)>::FunctionRef<realm::Query::do_count(unsigned long) const::$_6&>(realm::Query::do_count(unsigned long) const::$_6&)::'lambda'(void*, realm::Cluster const*)::__invoke(void*, realm::Cluster const*) () #6 0x0000000102b787ef in realm::Query::do_count(unsigned long) const () #7 0x0000000102b78bf9 in realm::Query::count(realm::DescriptorOrdering const&) () #8 0x0000000102a69a7f in realm::Results::do_size() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/results.cpp:156 #9 0x0000000102a6e789 in realm::Results::size() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/results.cpp:140 #10 0x0000000102b0df74 in -[RLMResults count]::$_1::operator()() const [inlined] at /Users/realm/workspace/cocoa-pipeline/Realm/RLMResults.mm:147 #11 0x0000000102b0df69 in auto translateRLMResultsErrors<-[RLMResults count]::$_1>(-[RLMResults count]::$_1&&, NSString*) [inlined] at /Users/realm/workspace/cocoa-pipeline/Realm/RLMResults_Private.hpp:60 #12 0x0000000102b0df69 in -[RLMResults count] at /Users/realm/workspace/cocoa-pipeline/Realm/RLMResults.mm:147 #13 0x000000010276c5e0 in Results.count.getter [inlined] at /Users/realm/workspace/cocoa-pipeline/RealmSwift/Results.swift:107 #14 0x000000010276c5d4 in Results.endIndex.getter at /Users/realm/workspace/cocoa-pipeline/RealmSwift/Results.swift:388 #15 0x000000010276cb72 in protocol witness for Collection.endIndex.getter in conformance Results<A> () #16 0x00007fff6897dbcf in Collection.isEmpty.getter () #17 0x000000010022a748 in closure #1 in AppCoordinator.updateFeaturedSectionsAfterSync() at /Users/allen.humphreys/personal_projects/WWDC/WWDC/AppCoordinator.swift:289 #18 0x000000010022a806 in thunk for @escaping @callee_guaranteed (@guaranteed Results<FeaturedSection>) -> (@unowned Bool, @error @owned Error) () #19 0x0000000100236354 in partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Results<FeaturedSection>) -> (@unowned Bool, @error @owned Error) () #20 0x00000001024750cf in FilterSink.on(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Filter.swift:58 #21 0x00000001024752b0 in protocol witness for ObserverType.on(_:) in conformance FilterSink<A> () #22 0x00000001024b0093 in Sink.forwardOn(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Sink.swift:34 #23 0x00000001024a75a5 in AnonymousObservableSink.on(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Create.swift:50 #24 0x00000001024a7690 in protocol witness for ObserverType.on(_:) in conformance AnonymousObservableSink<A> () #25 0x0000000102508273 in partial apply () #26 0x000000010248917c in AnyObserver.on(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/AnyObserver.swift:36 #27 0x0000000102508046 in ObserverType.onNext(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/ObserverType.swift:30 #28 0x000000010244498e in closure #1 in static ObservableType<>.collection(from:synchronousStart:on:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxRealm/Pod/Classes/RxRealm.swift:122 #29 0x0000000102447d12 in partial apply for closure #1 in static ObservableType<>.collection(from:synchronousStart:on:) () #30 0x0000000102447bd0 in partial apply for closure #1 in static ObservableType<>.collection(from:synchronousStart:on:) () #31 0x00000001024a77dc in AnonymousObservableSink.run(_:) [inlined] at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Create.swift:60 #32 0x00000001024a7794 in AnonymousObservable.run<A>(_:cancel:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Create.swift:75 #33 0x00000001024b554f in Producer.subscribe<A>(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Producer.swift:18 #34 0x00000001024753c1 in Filter.run<A>(_:cancel:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Filter.swift:87 #35 0x00000001024b554f in Producer.subscribe<A>(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Producer.swift:18 #36 0x0000000102508512 in protocol witness for ObservableType.subscribe<A>(_:) in conformance Observable<A> () #37 0x00000001024d8b0b in closure #1 in SubscribeOnSink.run() at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/SubscribeOn.swift:58 #38 0x00000001024d8f11 in partial apply for closure #1 in SubscribeOnSink.run() () #39 0x00000001024b56ac in thunk for @escaping @callee_guaranteed () -> (@out Disposable) () #40 0x00000001024d8f51 in partial apply for thunk for @escaping @callee_guaranteed () -> (@out Disposable) () #41 0x000000010248f72a in MainScheduler.scheduleInternal<A>(_:action:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Schedulers/MainScheduler.swift:63 #42 0x00000001024f61fc in SerialDispatchQueueScheduler.schedule<A>(_:action:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Schedulers/SerialDispatchQueueScheduler.swift:101 #43 0x00000001024f62c0 in protocol witness for ImmediateSchedulerType.schedule<A>(_:action:) in conformance SerialDispatchQueueScheduler () #44 0x00000001024d8a10 in SubscribeOnSink.run() at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/SubscribeOn.swift:57 #45 0x00000001024d8dd2 in SubscribeOn.run<A>(_:cancel:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/SubscribeOn.swift:80 #46 0x00000001024b554f in Producer.subscribe<A>(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Producer.swift:18 #47 0x00000001024ae7fd in TakeCount.run<A>(_:cancel:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Take.swift:102 #48 0x00000001024b5afe in closure #1 in Producer.subscribe<A>(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Producer.swift:26 #49 0x00000001024b5b99 in partial apply for closure #1 in Producer.subscribe<A>(_:) () #50 0x00000001024b5bd1 in thunk for @escaping @callee_guaranteed () -> (@out Disposable) [inlined] () #51 0x00000001024b5bce in partial apply for thunk for @escaping @callee_guaranteed () -> (@out Disposable) () #52 0x00000001024ad1b0 in CurrentThreadScheduler.schedule<A>(_:action:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Schedulers/CurrentThreadScheduler.swift:101 #53 0x00000001024b5648 in Producer.subscribe<A>(_:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/Observables/Producer.swift:24 #54 0x000000010249a236 in ObservableType.subscribe(onNext:onError:onCompleted:onDisposed:) at /Users/allen.humphreys/personal_projects/WWDC/Carthage/Checkouts/RxSwift/RxSwift/ObservableType+Extensions.swift:80 #55 0x000000010022a336 in AppCoordinator.updateFeaturedSectionsAfterSync() at /Users/allen.humphreys/personal_projects/WWDC/WWDC/AppCoordinator.swift:292 #56 0x000000010022df16 in AppCoordinator.startup() at /Users/allen.humphreys/personal_projects/WWDC/WWDC/AppCoordinator.swift:348 #57 0x00000001002d09bd in AppDelegate.startupUI(using:syncEngine:) at /Users/allen.humphreys/personal_projects/WWDC/WWDC/AppDelegate.swift:67 #58 0x00000001002cff26 in closure #2 in AppDelegate.applicationDidFinishLaunching(_:) at /Users/allen.humphreys/personal_projects/WWDC/WWDC/AppDelegate.swift:55 #59 0x00000001004aa16c in closure #1 in closure #2 in Boot.bootstrapDependencies(then:) at /Users/allen.humphreys/personal_projects/WWDC/WWDC/Boot.swift:100 #60 0x0000000100017593 in thunk for @escaping @callee_guaranteed () -> () () #61 0x0000000100cd5cdb in __wrap_dispatch_async_block_invoke () #62 0x0000000103459844 in _dispatch_call_block_and_release () #63 0x000000010345a826 in _dispatch_client_callout () #64 0x000000010346a446 in _dispatch_main_queue_callback_4CF () #65 0x00007fff2f38af11 in __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ () #66 0x00007fff2f34ad17 in __CFRunLoopRun () #67 0x00007fff2f349ece in CFRunLoopRunSpecific () #68 0x00007fff2df78abd in RunCurrentEventLoopInMode () #69 0x00007fff2df787d5 in ReceiveNextEventCommon () #70 0x00007fff2df78579 in _BlockUntilNextEventMatchingListInModeWithFilter () #71 0x00007fff2c5c0829 in _DPSNextEvent () #72 0x00007fff2c5bf070 in -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] () #73 0x00007fff2c5b0d7e in -[NSApplication run] () #74 0x00007fff2c582b86 in NSApplicationMain () #75 0x00000001000021b2 in main at /Users/allen.humphreys/personal_projects/WWDC/WWDC/main.m:12 #76 0x00007fff69346cc9 in start () #77 0x00007fff69346cc9 in start ()
</details>
<details><summary>Another Stack Track</summary>
Realm notification listener (16)#0 0x0000000102d166ed in long long realm::ConstObj::_get<long long>(realm::ColKey::Idx) const () #1 0x0000000102cd0804 in realm::ConstLstBase::get_child_ref(unsigned long) const () #2 0x0000000102a4707c in realm::BPlusTreeBase::init_from_parent() at /Users/realm/workspace/cocoa-pipeline/core/include/realm/bplustree.hpp:175 #3 0x0000000102cd787d in realm::Lst<realm::ObjKey>::Lst(realm::Obj const&, realm::ColKey) () #4 0x0000000102cd8d60 in realm::LnkLst::LnkLst(realm::Obj const&, realm::ColKey) () #5 0x0000000102cd869b in realm::LnkLst::clone() const () #6 0x0000000102b5df12 in realm::Query::Query(realm::Query const&) () #7 0x0000000102b7bddd in realm::ConstTableView::ConstTableView(realm::ConstTableRef, realm::Query&, unsigned long, unsigned long, unsigned long) () #8 0x0000000102b754bb in realm::Query::find_all(unsigned long, unsigned long, unsigned long) () #9 0x0000000102a7445f in realm::_impl::ResultsNotifier::run() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/results_notifier.cpp:158 #10 0x0000000102a605be in realm::_impl::RealmCoordinator::run_async_notifiers() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/realm_coordinator.cpp:994 #11 0x0000000102a5fe56 in realm::_impl::RealmCoordinator::on_change() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/realm_coordinator.cpp:782 #12 0x0000000102a39d71 in realm::_impl::ExternalCommitHelper::listen() at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/apple/external_commit_helper.cpp:231 #13 0x0000000102a3a302 in realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0::operator()() const [inlined] at /Users/realm/workspace/cocoa-pipeline/Realm/ObjectStore/src/impl/apple/external_commit_helper.cpp:173 #14 0x0000000102a3a2fd in decltype(std::__1::forward<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0>(fp)()) std::__1::__invoke<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0>(realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0&&) [inlined] at /Applications/Xcode-11.5.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/type_traits:4425 #15 0x0000000102a3a2fd in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0>&, std::__1::__tuple_indices<>) [inlined] at /Applications/Xcode-11.5.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/thread:341 #16 0x0000000102a3a2fd in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >(void*) at /Applications/Xcode-11.5.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/thread:351 #17 0x00000001034e8c65 in _pthread_start () #18 0x00000001034e44af in thread_start ()
</details>
Steps to Reproduce
The production steps are quite simple and I think it's basically 100%
- Use version 7.0.2 of WWDC.io. Be sure to leave the application open for about a minute or so an maybe scroll through the tables to populate some images
- Build and run from our branch that uses version 5 of Realm and always returns true from the shouldCompactOnLaunch closure
- crash
Code Sample
https://github.com/insidegui/WWDC/compare/master...ah/realm-5-plus-always-compact
Version of Realm and Tooling
<!---
In the CONTRIBUTING guidelines, you will find a script,
which will help determining some of these versions.
-->
Realm framework version: v5.3.0
Realm Object Server version: N/A
Xcode version: 11.5 (11E608c)
OSX version: 10.15.5 (19F101)
Dependency manager + version: Carthage + 0.34.0