Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-2260

Thread blocked forever when operating on ReplicaSet

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.2
    • Component/s: None
    • None

      I use mongodb java driver 3.3.0. When the Java code operating on ReplicaSet, whatever query or update, many threads blocked and never return. Use jstack to observe the threads, it's like:

      "catalina-exec-268" #354 daemon prio=5 os_prio=0 tid=0x00007fe0d8052000 nid=0x6553 waiting on condition [0x00007fe0934c2000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <0x00000007a36970b0> (a java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
        at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:250)
        at com.mongodb.connection.UsageTrackingInternalConnection.receiveMessage(UsageTrackingInternalConnection.java:105)
        at com.mongodb.connection.DefaultConnectionPool$PooledConnection.receiveMessage(DefaultConnectionPool.java:438)
        at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:112)
        at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)
        at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289)
        at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:176)
        at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:216)
        at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:207)
        at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:113)
        at com.mongodb.operation.FindOperation$1.call(FindOperation.java:488)
        at com.mongodb.operation.FindOperation$1.call(FindOperation.java:483)
        at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:241)
        at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:214)
        at com.mongodb.operation.FindOperation.execute(FindOperation.java:483)
        at com.mongodb.operation.FindOperation.execute(FindOperation.java:80)
        at com.mongodb.Mongo.execute(Mongo.java:818)
        at com.mongodb.Mongo$2.execute(Mongo.java:805)
        at com.mongodb.DBCollection.findOne(DBCollection.java:777)
        at com.mongodb.DBCollection.findOne(DBCollection.java:747)
        at com.mongodb.DBCollection.findOne(DBCollection.java:694)
        at com.bugull.mongo.BuguDao.findOne(BuguDao.java:588)
        at com.bugull.farm.web.module.pad.PadApiController.place(PadApiController.java:344)
        at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:781)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:721)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at com.bugull.farm.web.filter.ApiFilter.doFilter(ApiFilter.java:42)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
        at org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:194)
        at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:318)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
      • locked <0x00000007bb13a0d0> (a org.apache.tomcat.util.net.NioChannel)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)

      Lots of theads are WAITING. Read the attachment for more details.
      I change driver to 3.2.2, problems are still the same. Then I change driver to 2.14.3, everything works fine.
      If connected to a single mongodb server, instead of replica set, all the driver works fine.
      So I think it's a bug of the 3.x version.
      I use JDK 8u71 and Tomcat 8.0.30. Mongodb server is 3.2.0.

        1. jstack_tomcat.txt
          725 kB
          Frank Wen

            Assignee:
            Unassigned Unassigned
            Reporter:
            xbwen Frank Wen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: