Loading...

XML

Word

Printable

JSON

Type: Bug
Resolution: Duplicate
Priority: Blocker - P1
Fix Version/s: None
Affects Version/s: 2.11.5
Component/s: Connectivity
Labels:
None
Environment:
.NET Framework 4.8 on Windows Server 2019
.NET Core 3.1 on Ubuntu 18.04 x64

First of all, I kindly ask you to take a very close look at this. It is really important and can kill production systems in minutes. I believe this should be fixed and released for the 2.11 series as soon as possible.

There appears to be a deadlock in the connection management code that was introduced in the 2.11 driver series.
Due to the unfortunate decision not to ship debug symbols with the NuGet package, diagnosing it took more time than it should, but, well, let's take a look.

The problem manifests itself as a semi-permanent timeout (System.TimeoutException) for every operation directed to a MongoDB host. It is triggered by the host being restarted, thus being unavailable for heartbeats for a short time. Unfortunately, after the host gets up and running, not all clients successfully recover from this issue, with some of them entering the timeout of death state. Given that the issue appears to have a non-100% probability of occurrence did lead us to the idea that there is some race condition behind this. It occurs more frequently on .NET Core 3.1 running on Linux, probably due to how locking is implemented there, but .NET Framework 4.8 running on Windows is susceptible as well (in fact last time it occurred it killed about 40% of our backends).

Without much further mingling on the matter, let me present you a couple of stack traces. The first (thread 7256) is the same for about 1200 other threads, waiting on a lock held by thread 1400, which is in the second stack trace. It appears that parallel heartbeats (background vs directly requested) can lock these locks out of order since the heartbeat process itself (server description change handler) can try to request another heartbeat while being under the lock.

Thread 7256 is trying to lock the
MultiServerCluster._updateClusterDescriptionLock at MultiServerCluster.cs:278, while thread 1400 is trying to get a hold of a ServerMonitor._lock ServerMonitor.cs:146.

Thread 7256:

 	[Waiting on lock owned by Thread 1400, double-click or press enter to switch to thread]	
 	[Managed to Native Transition]	
>	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Clusters.MultiServerCluster.ProcessServerDescriptionChanged(MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs args)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Clusters.MultiServerCluster.ServerDescriptionChangedHandler(object sender, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs args)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.Server.OnDescriptionChanged(object sender, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs e)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.Server.OnMonitorDescriptionChanged(object sender, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs e)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.OnDescriptionChanged(MongoDB.Driver.Core.Servers.ServerDescription oldDescription, MongoDB.Driver.Core.Servers.ServerDescription newDescription)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.HeartbeatAsync(System.Threading.CancellationToken cancellationToken)	Unknown
 	[Resuming Async Method]	
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Driver.Core.Connections.IsMasterResult>.SetResult(MongoDB.Driver.Core.Connections.IsMasterResult result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.GetIsMasterResultAsync(MongoDB.Driver.Core.Connections.IConnection connection, MongoDB.Driver.Core.WireProtocol.CommandWireProtocol<MongoDB.Bson.BsonDocument> isMasterProtocol, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Driver.Core.Connections.IsMasterResult>.SetResult(MongoDB.Driver.Core.Connections.IsMasterResult result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.IsMasterHelper.GetResultAsync(MongoDB.Driver.Core.Connections.IConnection connection, MongoDB.Driver.Core.WireProtocol.CommandWireProtocol<MongoDB.Bson.BsonDocument> isMasterProtocol, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Bson.BsonDocument>.SetResult(MongoDB.Bson.BsonDocument result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol<MongoDB.Bson.BsonDocument>.ExecuteAsync(MongoDB.Driver.Core.Connections.IConnection connection, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Driver.Core.WireProtocol.Messages.ResponseMessage>.SetResult(MongoDB.Driver.Core.WireProtocol.Messages.ResponseMessage result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveMessageAsync(int responseTo, MongoDB.Driver.Core.WireProtocol.Messages.Encoders.IMessageEncoderSelector encoderSelector, MongoDB.Driver.Core.WireProtocol.Messages.Encoders.MessageEncoderSettings messageEncoderSettings, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Bson.IO.IByteBuffer>.SetResult(MongoDB.Bson.IO.IByteBuffer result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBufferAsync(int responseTo, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Bson.IO.IByteBuffer>.SetResult(MongoDB.Bson.IO.IByteBuffer result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBufferAsync(System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult>.TrySetResult(System.Threading.Tasks.VoidTaskResult result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetResult(System.Threading.Tasks.VoidTaskResult result) Line 603	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetResult(System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult> completedTask) Line 638	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Misc.StreamExtensionMethods.ReadBytesAsync(System.IO.Stream stream, MongoDB.Bson.IO.IByteBuffer buffer, int offset, int count, System.TimeSpan timeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>.SetResult(int result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Misc.StreamExtensionMethods.ReadAsync(System.IO.Stream stream, byte[] buffer, int offset, int count, System.TimeSpan timeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 490	C#
 	mscorlib.dll!System.Threading.Tasks.TaskFactory<int>.FromAsyncTrimPromise<System.IO.Stream>.Complete(System.IO.Stream thisRef, System.Func<System.IO.Stream, System.IAsyncResult, int> endMethod, System.IAsyncResult asyncResult, bool requiresSynchronization) Line 1438	C#
 	mscorlib.dll!System.Threading.Tasks.TaskFactory<int>.FromAsyncTrimPromise<System.IO.Stream>.CompleteFromAsyncResult(System.IAsyncResult asyncResult) Line 1412	C#
 	System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 917	C#
 	System.dll!System.Net.ContextAwareResult.Complete(System.IntPtr userToken)	Unknown
 	System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken)	Unknown
 	System.dll!System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped)	Unknown
 	mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) Line 135	C#
 	[Async Call Stack]	
 	[Async] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.MonitorServerAsync()	Unknown

Thread 1400:

 	[Waiting on lock owned by Thread 7256, double-click or press enter to switch to thread]	
 	[Managed to Native Transition]	
>	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.RequestHeartbeat()	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.Server.Invalidate(string reasonInvalidated, bool clearConnectionPool, MongoDB.Driver.Core.Servers.TopologyVersion responseTopologyVersion)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Clusters.MultiServerCluster.ProcessReplicaSetChange(MongoDB.Driver.Core.Clusters.ClusterDescription clusterDescription, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs args, System.Collections.Generic.List<MongoDB.Driver.Core.Servers.IClusterableServer> newServers)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Clusters.MultiServerCluster.ProcessServerDescriptionChanged(MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs args)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Clusters.MultiServerCluster.ServerDescriptionChangedHandler(object sender, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs args)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.Server.OnDescriptionChanged(object sender, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs e)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.Server.OnMonitorDescriptionChanged(object sender, MongoDB.Driver.Core.Servers.ServerDescriptionChangedEventArgs e)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.OnDescriptionChanged(MongoDB.Driver.Core.Servers.ServerDescription oldDescription, MongoDB.Driver.Core.Servers.ServerDescription newDescription)	Unknown
 	MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.HeartbeatAsync(System.Threading.CancellationToken cancellationToken)	Unknown
 	[Resuming Async Method]	
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Driver.Core.Connections.IsMasterResult>.SetResult(MongoDB.Driver.Core.Connections.IsMasterResult result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.GetIsMasterResultAsync(MongoDB.Driver.Core.Connections.IConnection connection, MongoDB.Driver.Core.WireProtocol.CommandWireProtocol<MongoDB.Bson.BsonDocument> isMasterProtocol, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Driver.Core.Connections.IsMasterResult>.SetResult(MongoDB.Driver.Core.Connections.IsMasterResult result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.IsMasterHelper.GetResultAsync(MongoDB.Driver.Core.Connections.IConnection connection, MongoDB.Driver.Core.WireProtocol.CommandWireProtocol<MongoDB.Bson.BsonDocument> isMasterProtocol, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Bson.BsonDocument>.SetResult(MongoDB.Bson.BsonDocument result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol<MongoDB.Bson.BsonDocument>.ExecuteAsync(MongoDB.Driver.Core.Connections.IConnection connection, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Driver.Core.WireProtocol.Messages.ResponseMessage>.SetResult(MongoDB.Driver.Core.WireProtocol.Messages.ResponseMessage result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveMessageAsync(int responseTo, MongoDB.Driver.Core.WireProtocol.Messages.Encoders.IMessageEncoderSelector encoderSelector, MongoDB.Driver.Core.WireProtocol.Messages.Encoders.MessageEncoderSettings messageEncoderSettings, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Bson.IO.IByteBuffer>.SetResult(MongoDB.Bson.IO.IByteBuffer result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBufferAsync(int responseTo, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<MongoDB.Bson.IO.IByteBuffer>.SetResult(MongoDB.Bson.IO.IByteBuffer result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBufferAsync(System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult>.TrySetResult(System.Threading.Tasks.VoidTaskResult result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetResult(System.Threading.Tasks.VoidTaskResult result) Line 603	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetResult(System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult> completedTask) Line 638	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Misc.StreamExtensionMethods.ReadBytesAsync(System.IO.Stream stream, MongoDB.Bson.IO.IByteBuffer buffer, int offset, int count, System.TimeSpan timeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 490	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>.SetResult(int result) Line 603	C#
 	[Completed] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Misc.StreamExtensionMethods.ReadAsync(System.IO.Stream stream, byte[] buffer, int offset, int count, System.TimeSpan timeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Line 1070	C#
 	mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Line 810	C#
 	mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Line 3617	C#
 	mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Line 490	C#
 	mscorlib.dll!System.Threading.Tasks.TaskFactory<int>.FromAsyncTrimPromise<System.IO.Stream>.Complete(System.IO.Stream thisRef, System.Func<System.IO.Stream, System.IAsyncResult, int> endMethod, System.IAsyncResult asyncResult, bool requiresSynchronization) Line 1438	C#
 	mscorlib.dll!System.Threading.Tasks.TaskFactory<int>.FromAsyncTrimPromise<System.IO.Stream>.CompleteFromAsyncResult(System.IAsyncResult asyncResult) Line 1412	C#
 	System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928	C#
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 917	C#
 	System.dll!System.Net.ContextAwareResult.Complete(System.IntPtr userToken)	Unknown
 	System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken)	Unknown
 	System.dll!System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped)	Unknown
 	mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) Line 135	C#
 	[Async Call Stack]	
 	[Async] MongoDB.Driver.Core.dll!MongoDB.Driver.Core.Servers.ServerMonitor.MonitorServerAsync()	Unknown

duplicates

CSHARP-3302 SDAM deadlock when invalidating former primary

Closed

Assignee:: Unassigned

Reporter:: Aristarkh Zagorodnikov

Votes:: 1 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: Jan 11 2021 11:42:42 AM UTC

Updated:: Sep 13 2021 05:18:07 PM UTC

Resolved:: Jan 11 2021 03:32:22 PM UTC

Details

Description

Attachments

Issue Links

Activity

People

Dates