-
Type: Bug
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
Fully Compatible
-
ALL
-
Sharding 2021-02-22
-
59
One ASAN example:
[js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:46.179+0000 d20020| ==113923==ERROR: AddressSanitizer: heap-use-after-free on address 0x607000574ad0 at pc 0x5606994700ea bp 0x7f1e80775650 sp 0x7f1e80774dd8 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:46.179+0000 d20020| READ of size 1 at 0x607000574ad0 thread T373 (TenantM.vice-59) [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.275+0000 d20020| #0 0x5606994700e9 in strcmp /data/mci/a0a92963d3836bdfa3c974d486c10ee5/toolchain-builder/tmp/build-llvm.sh-DWL/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:432:3 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.276+0000 d20020| #1 0x7f1f4da3abb8 in OBJ_bsearch_ex_ (/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1+0x185bb8) [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.277+0000 d20020| #2 0x7f1f4da3b241 in OBJ_ln2nid (/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1+0x186241) [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.277+0000 d20020| #3 0x7f1f4da3b365 in OBJ_txt2obj (/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1+0x186365) [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.277+0000 d20020| #4 0x7f1f4da3b49c in OBJ_txt2nid (/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1+0x18649c) [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.301+0000 d20020| #5 0x5606a226dc9e in mongo::x509OidToShortName[abi:cxx11](mongo::StringData) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_manager.cpp:464:22 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.301+0000 d20020| #6 0x5606a22709f8 in mongo::operator<<(mongo::StringBuilderImpl<mongo::BufBuilder>&, mongo::SSLX509Name const&) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_manager.cpp:697:27 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.302+0000 d20020| #7 0x5606a2261f30 in mongo::SSLX509Name::toString[abi:cxx11]() const /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_manager.cpp:707:8 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.321+0000 d20020| #8 0x5606a1cf3af1 in mongo::DBClientBase::_auth(mongo::BSONObj const&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/client/dbclient_base.cpp:510:58 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.338+0000 d20020| #9 0x5606a1276256 in mongo::DBClientConnection::_auth(mongo::BSONObj const&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/client/dbclient_connection.cpp:264:19 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.395+0000 d20020| #10 0x56069b3f5ad7 in mongo::repl::TenantMigrationRecipientService::Instance::_connectAndAuth(mongo::HostAndPort const&, mongo::StringData, mongo::TransientSSLParams const*) /data/mci/285b95a2e8a76dc69eb920a5ed416a5a/src/src/mongo/db/repl/tenant_migration_recipient_service.cpp:316:17 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.396+0000 d20020| #11 0x56069b437cef in mongo::repl::TenantMigrationRecipientService::Instance::_createAndConnectClients()::$_4::operator()(mongo::HostAndPort const&) const /data/mci/285b95a2e8a76dc69eb920a5ed416a5a/src/src/mongo/db/repl/tenant_migration_recipient_service.cpp:384:27 ... [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| 0x607000574ad0 is located 16 bytes inside of 72-byte region [0x607000574ac0,0x607000574b08) [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| freed by thread T370 (TenantM.vice-56) here: [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| #0 0x560699533792 in operator delete(void*) /data/mci/a0a92963d3836bdfa3c974d486c10ee5/toolchain-builder/tmp/build-llvm.sh-DWL/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:167:3 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| #1 0x5606a228a46a in __gnu_cxx::new_allocator<mongo::SSLX509Name::Entry>::deallocate(mongo::SSLX509Name::Entry*, unsigned long) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/ext/new_allocator.h:125:2 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| #2 0x5606a228a46a in std::allocator_traits<std::allocator<mongo::SSLX509Name::Entry> >::deallocate(std::allocator<mongo::SSLX509Name::Entry>&, mongo::SSLX509Name::Entry*, unsigned long) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/alloc_traits.h:462 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| #3 0x5606a228a46a in std::_Vector_base<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >::_M_deallocate(mongo::SSLX509Name::Entry*, unsigned long) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_vector.h:304 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.492+0000 d20020| #4 0x5606a228a46a in std::_Vector_base<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >::~_Vector_base() /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_vector.h:285 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #5 0x5606a228a46a in std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >::~vector() /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_vector.h:570 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #6 0x5606a228a46a in void std::_Destroy<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > >(std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_construct.h:98 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #7 0x5606a228a46a in void std::_Destroy_aux<false>::__destroy<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*>(std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*, std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_construct.h:108 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #8 0x5606a228a46a in void std::_Destroy<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*>(std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*, std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_construct.h:136 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #9 0x5606a228a46a in void std::_Destroy<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*, std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > >(std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*, std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >*, std::allocator<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > >&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_construct.h:206 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #10 0x5606a228a46a in std::vector<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >, std::allocator<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > > >::~vector() /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_vector.h:567 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.493+0000 d20020| #11 0x5606a228a46a in std::vector<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >, std::allocator<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > > >::_M_move_assign(std::vector<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >, std::allocator<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > > >&&, std::integral_constant<bool, true>) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_vector.h:1683 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.563+0000 d20020| #12 0x5606a22f99f8 in std::vector<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >, std::allocator<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > > >::operator=(std::vector<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> >, std::allocator<std::vector<mongo::SSLX509Name::Entry, std::allocator<mongo::SSLX509Name::Entry> > > >&&) /opt/mongodbtoolchain/revisions/39699409944dd532c7cbdce2f62da11361ac220a/stow/gcc-v3.Me0/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/stl_vector.h:601:2 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.563+0000 d20020| #13 0x5606a22f99f8 in mongo::SSLX509Name::operator=(mongo::SSLX509Name&&) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_types.h:51 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.563+0000 d20020| #14 0x5606a22f99f8 in mongo::(anonymous namespace)::SSLManagerOpenSSL::_parseAndValidateCertificateFromBIO(std::unique_ptr<bio_st, mongo::OpenSSLDeleter<int (bio_st*), &(BIO_free)> >, mongo::(anonymous namespace)::SSLManagerOpenSSL::PasswordFetcher*, mongo::StringData, mongo::SSLX509Name*, mongo::Date_t*) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_manager_openssl.cpp:2393 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.563+0000 d20020| #15 0x5606a22d7003 in mongo::(anonymous namespace)::SSLManagerOpenSSL::_parseAndValidateCertificateFromMemory(mongo::StringData, mongo::(anonymous namespace)::SSLManagerOpenSSL::PasswordFetcher*, mongo::SSLX509Name*, mongo::Date_t*) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_manager_openssl.cpp:2370:12 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.563+0000 d20020| #16 0x5606a22d7003 in mongo::(anonymous namespace)::SSLManagerOpenSSL::initSSLContext(ssl_ctx_st*, mongo::SSLParams const&, mongo::TransientSSLParams const&, mongo::SSLManagerInterface::ConnectionDirection) /data/mci/00a3370018bfacd9240233748dd9166f/src/src/mongo/util/net/ssl_manager_openssl.cpp:2214 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.637+0000 d20020| #17 0x5606a15f5e50 in mongo::transport::TransportLayerASIO::_createSSLContext(std::shared_ptr<mongo::SSLManagerInterface>&, mongo::SSLParams::SSLModes, mongo::TransientSSLParams, bool) const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/transport/transport_layer_asio.cpp:1283:49 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.637+0000 d20020| #18 0x5606a15f8046 in mongo::transport::TransportLayerASIO::createTransientSSLContext(mongo::TransientSSLParams const&) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/transport/transport_layer_asio.cpp:1308:12 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.637+0000 d20020| #19 0x5606a16107f4 in mongo::StatusWith<std::shared_ptr<mongo::transport::TransportLayerASIO::ASIOSession> > mongo::transport::TransportLayerASIO::_doSyncConnect<mongo::transport::WrappedEndpoint>(mongo::transport::WrappedEndpoint, mongo::HostAndPort const&, mongo::Duration<std::ratio<1l, 1000l> > const&, boost::optional<mongo::TransientSSLParams>) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/transport/transport_layer_asio.cpp:582:36 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.637+0000 d20020| #20 0x5606a15d8255 in mongo::transport::TransportLayerASIO::connect(mongo::HostAndPort, mongo::transport::ConnectSSLMode, mongo::Duration<std::ratio<1l, 1000l> >, boost::optional<mongo::TransientSSLParams>) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/transport/transport_layer_asio.cpp:486:16 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.640+0000 d20020| #21 0x5606a159e5ba in mongo::transport::TransportLayerManager::connect(mongo::HostAndPort, mongo::transport::ConnectSSLMode, mongo::Duration<std::ratio<1l, 1000l> >, boost::optional<mongo::TransientSSLParams>) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/transport/transport_layer_manager.cpp:69:26 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.640+0000 d20020| #22 0x5606a12840d1 in mongo::DBClientConnection::connectSocketOnly(mongo::HostAndPort const&, boost::optional<mongo::TransientSSLParams>) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/client/dbclient_connection.cpp:422:64 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.641+0000 d20020| #23 0x5606a127742e in mongo::DBClientConnection::connect(mongo::HostAndPort const&, mongo::StringData, boost::optional<mongo::TransientSSLParams>) /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/client/dbclient_connection.cpp:291:26 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.644+0000 d20020| #24 0x5606a12514f1 in mongo::ConnectionString::connect(mongo::StringData, double, mongo::MongoURI const*, mongo::ClientAPIVersionParameters const*, mongo::TransientSSLParams const*) const /data/mci/3da64909ec0a2680321efc4ca3c244ba/src/src/mongo/client/connection_string_connect.cpp:75:32 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.644+0000 d20020| #25 0x56069b3f53a0 in mongo::repl::TenantMigrationRecipientService::Instance::_connectAndAuth(mongo::HostAndPort const&, mongo::StringData, mongo::TransientSSLParams const*) /data/mci/285b95a2e8a76dc69eb920a5ed416a5a/src/src/mongo/db/repl/tenant_migration_recipient_service.cpp:294:30 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.644+0000 d20020| #26 0x56069b437fc8 in mongo::repl::TenantMigrationRecipientService::Instance::_createAndConnectClients()::$_4::operator()(mongo::HostAndPort const&) const /data/mci/285b95a2e8a76dc69eb920a5ed416a5a/src/src/mongo/db/repl/tenant_migration_recipient_service.cpp:392:17 [js_test:tenant_migration_concurrent_migrations_recipient] 2021-01-27T20:05:49.644+0000 d20020| #27 0x56069b437fc8 in std::pair<std::unique_ptr<mongo::DBClientConnection, std::default_delete<mongo::DBClientConnection> >, std::unique_ptr<mongo::DBClientConnection, std::default_delete<mongo::DBClientConnection> > > mongo::unique_function<std::pair<std::unique_ptr<mongo::DBClientConnection, std::default_delete<mongo::DBClientConnection> >, std::unique_ptr<mongo::DBClientConnection, std::default_delete<mongo::DBClientConnection> > > (mongo::HostAndPort)>::callRegularVoid<mongo::repl::TenantMigrationRecipientService::Instance::_createAndConnectClients()::$_4>(std::integral_constant<bool, false>, mongo::repl::TenantMigrationRecipientService::Instance::_createAndConnectClients()::$_4&, mongo::HostAndPort&&) /data/mci/285b95a2e8a76dc69eb920a5ed416a5a/src/src/mongo/util/functional.h:150
- depends on
-
SERVER-54328 Refactor creation of transient SSLConnectionContext to own its own instance of SSLManagerInterface
- Closed
- is depended on by
-
SERVER-53168 Support 50 concurrent migrations on a single recipient
- Closed
- is duplicated by
-
SERVER-54108 ASAN failure in SSL manager
- Closed