2018-11-14 23:59:55,002 INFO jobs-thread-3 task.base.JobExecuteTemplate.triggerJob(JobExecuteTemplate.java:26) ~ JobExecuteTemplate.java TASK >>>>>>>>>>>Job [HandleExpressStateRecordJob] trigger.Job Id = 1542211195002 2018-11-14 23:59:55,003 INFO jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.cluster No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=172.25.164.82:3717, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 0]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=2607069, setName='mgset-400006496', canonicalAddress=172.25.164.83:3717, hosts=[172.25.164.83:3717, 172.25.164.82:3717], passives=[], arbiters=[], primary='null', tagSet=TagSet{[]}, electionId=null, setVersion=1, lastWriteDate=Wed Nov 14 23:59:47 CST 2018, lastUpdateTimeNanos=184209739247471}]}. Waiting for 30000 ms before timing out 2018-11-14 23:59:55,003 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Checking status of 172.25.164.82:3717 2018-11-14 23:59:55,004 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Sending command '{ "ismaster" : 1, "$db" : "admin", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1542211187, "i" : 1 } }, "signature" : { "hash" : { "$binary" : { "base64" : "jpgZ8pJXzzJHWByKj/G82MUv05A=", "subType" : "00" } }, "keyId" : 6621811776168132609 } } }' with request id 44 to database admin on connection [connectionId{localValue:20, serverValue:316526}] to server 172.25.164.82:3717 2018-11-14 23:59:55,004 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Checking status of 172.25.164.83:3717 2018-11-14 23:59:55,002 INFO jobs-thread-1 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.cluster No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=172.25.164.82:3717, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 0]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=2607069, setName='mgset-400006496', canonicalAddress=172.25.164.83:3717, hosts=[172.25.164.83:3717, 172.25.164.82:3717], passives=[], arbiters=[], primary='null', tagSet=TagSet{[]}, electionId=null, setVersion=1, lastWriteDate=Wed Nov 14 23:59:47 CST 2018, lastUpdateTimeNanos=184209739247471}]}. Waiting for 30000 ms before timing out 2018-11-14 23:59:55,005 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Sending command '{ "ismaster" : 1, "$db" : "admin", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1542211187, "i" : 1 } }, "signature" : { "hash" : { "$binary" : { "base64" : "jpgZ8pJXzzJHWByKj/G82MUv05A=", "subType" : "00" } }, "keyId" : 6621811776168132609 } } }' with request id 45 to database admin on connection [connectionId{localValue:3, serverValue:247323}] to server 172.25.164.83:3717 2018-11-14 23:59:55,005 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Execution of command with request id 44 completed successfully in 1.96 ms on connection [connectionId{localValue:20, serverValue:316526}] to server 172.25.164.82:3717 2018-11-14 23:59:55,005 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Execution of command with request id 44 completed successfully in 1.96 ms on connection [connectionId{localValue:20, serverValue:316526}] to server 172.25.164.82:3717 2018-11-14 23:59:55,006 INFO cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.cluster Discovered replica set primary 172.25.164.82:3717 2018-11-14 23:59:55,006 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Updating cluster description to {type=REPLICA_SET, servers=[{address=172.25.164.82:3717, type=REPLICA_SET_PRIMARY, roundTripTime=1.8 ms, state=CONNECTED}, {address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, roundTripTime=2.6 ms, state=CONNECTED}] 2018-11-14 23:59:55,006 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Execution of command with request id 45 completed successfully in 1.38 ms on connection [connectionId{localValue:3, serverValue:247323}] to server 172.25.164.83:3717 2018-11-14 23:59:55,007 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Updating cluster description to {type=REPLICA_SET, servers=[{address=172.25.164.82:3717, type=REPLICA_SET_PRIMARY, roundTripTime=1.8 ms, state=CONNECTED}, {address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, roundTripTime=2.6 ms, state=CONNECTED}] 2018-11-14 23:59:55,007 DEBUG jobs-thread-1 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.connection Closing connection connectionId{localValue:25} 2018-11-14 23:59:55,008 INFO jobs-thread-1 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.connection Closed connection [connectionId{localValue:25}] to 172.25.164.82:3717 because there was a socket exception raised by this connection. 2018-11-14 23:59:55,008 DEBUG jobs-thread-1 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Updating cluster description to {type=REPLICA_SET, servers=[{address=172.25.164.82:3717, type=UNKNOWN, state=CONNECTING}, {address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, roundTripTime=2.6 ms, state=CONNECTED}] 2018-11-14 23:59:55,008 DEBUG jobs-thread-1 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.connection Invalidating the connection pool 2018-11-14 23:59:55,009 ERROR jobs-thread-1 play.Logger.niceThrowable(Logger.java:604) ~ Logger.java play @7a08ikh4m Error during job execution (task.TempJob) Execution exception (In /app/utils/MongoUtils.java around line 81) MongoSocketReadException occurred : Prematurely reached end of stream play.exceptions.JavaExecutionException: Prematurely reached end of stream at play.jobs.Job.call(Job.java:225) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: com.mongodb.MongoSocketReadException: Prematurely reached end of stream at com.mongodb.internal.connection.SocketChannelStream.read(SocketChannelStream.java:96) at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:554) at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:425) at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255) at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83) at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33) at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:106) at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:63) at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:127) at com.mongodb.internal.connection.UsageTrackingInternalConnection.open(UsageTrackingInternalConnection.java:50) at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.open(DefaultConnectionPool.java:390) at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:106) at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:92) at com.mongodb.internal.connection.DefaultServer.getConnection(DefaultServer.java:85) at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.getConnection(ClusterBinding.java:114) at com.mongodb.client.internal.ClientSessionBinding$SessionBindingConnectionSource.getConnection(ClientSessionBinding.java:111) at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:460) at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:406) at com.mongodb.operation.FindOperation.execute(FindOperation.java:695) at com.mongodb.operation.FindOperation.execute(FindOperation.java:83) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:179) at com.mongodb.client.internal.FindIterableImpl.first(FindIterableImpl.java:198) at utils.MongoUtils.findFirst(MongoUtils.java:81) at task.base.JobExecuteTemplate.findByName(JobExecuteTemplate.java:47) at task.base.JobExecuteTemplate.triggerJob(JobExecuteTemplate.java:28) at task.TempJob.doJob(TempJob.java:26) at play.jobs.Job.doJobWithResult(Job.java:54) at play.jobs.Job$2.apply(Job.java:208) at play.db.jpa.JPA.withTransaction(JPA.java:316) at play.db.jpa.JPA.withinFilter(JPA.java:212) at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:292) at play.jobs.Job.withinFilter(Job.java:185) at play.jobs.Job.call(Job.java:204) ... 6 more 2018-11-14 23:59:55,095 INFO jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.connection Opened connection [connectionId{localValue:24, serverValue:316532}] to 172.25.164.82:3717 2018-11-14 23:59:55,102 DEBUG jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Sending command '{ "find" : "JobModel", "filter" : { "jobName" : "HandleExpressStateRecordJob", "isDelete" : 0 }, "limit" : 1, "singleBatch" : true, "$db" : "kikuu", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1542211187, "i" : 1 } }, "signature" : { "hash" : { "$binary" : { "base64" : "jpgZ8pJXzzJHWByKj/G82MUv05A=", "subType" : "00" } }, "keyId" : 6621811776168132609 } }, "lsid" : { "id" : { "$binary" : { "base64" : "qlozHeEbRoqK6+4+Y1bBrA==", "subType" : "04" } } } }' with request id 53 to database kikuu on connection [connectionId{localValue:24, serverValue:316532}] to server 172.25.164.82:3717 2018-11-14 23:59:55,103 DEBUG jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Execution of command with request id 53 completed successfully in 1.58 ms on connection [connectionId{localValue:24, serverValue:316532}] to server 172.25.164.82:3717 2018-11-14 23:59:55,106 INFO jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.connection Closed connection [connectionId{localValue:24, serverValue:316532}] to 172.25.164.82:3717 because there was a socket exception raised on another connection from this pool. 2018-11-14 23:59:55,106 DEBUG jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.connection Closing connection connectionId{localValue:24, serverValue:316532} 2018-11-14 23:59:55,118 INFO jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.cluster No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=172.25.164.82:3717, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 0]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=2564015, setName='mgset-400006496', canonicalAddress=172.25.164.83:3717, hosts=[172.25.164.83:3717, 172.25.164.82:3717], passives=[], arbiters=[], primary='null', tagSet=TagSet{[]}, electionId=null, setVersion=1, lastWriteDate=Wed Nov 14 23:59:47 CST 2018, lastUpdateTimeNanos=184211234946993}]}. Waiting for 30000 ms before timing out 2018-11-14 23:59:55,506 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Checking status of 172.25.164.82:3717 2018-11-14 23:59:55,507 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Sending command '{ "ismaster" : 1, "$db" : "admin", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1542211187, "i" : 1 } }, "signature" : { "hash" : { "$binary" : { "base64" : "jpgZ8pJXzzJHWByKj/G82MUv05A=", "subType" : "00" } }, "keyId" : 6621811776168132609 } } }' with request id 54 to database admin on connection [connectionId{localValue:20, serverValue:316526}] to server 172.25.164.82:3717 2018-11-14 23:59:55,507 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Checking status of 172.25.164.83:3717 2018-11-14 23:59:55,508 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Sending command '{ "ismaster" : 1, "$db" : "admin", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1542211187, "i" : 1 } }, "signature" : { "hash" : { "$binary" : { "base64" : "jpgZ8pJXzzJHWByKj/G82MUv05A=", "subType" : "00" } }, "keyId" : 6621811776168132609 } } }' with request id 55 to database admin on connection [connectionId{localValue:3, serverValue:247323}] to server 172.25.164.83:3717 2018-11-14 23:59:55,508 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Execution of command with request id 54 completed successfully in 1.79 ms on connection [connectionId{localValue:20, serverValue:316526}] to server 172.25.164.82:3717 2018-11-14 23:59:55,509 INFO cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.cluster Discovered replica set primary 172.25.164.82:3717 2018-11-14 23:59:55,509 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.protocol.command Execution of command with request id 55 completed successfully in 1.52 ms on connection [connectionId{localValue:3, serverValue:247323}] to server 172.25.164.83:3717 2018-11-14 23:59:55,509 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.82:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Updating cluster description to {type=REPLICA_SET, servers=[{address=172.25.164.82:3717, type=REPLICA_SET_PRIMARY, roundTripTime=2.0 ms, state=CONNECTED}, {address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, roundTripTime=2.6 ms, state=CONNECTED}] 2018-11-14 23:59:55,510 DEBUG cluster-ClusterId{value='5bec4675306e3c0cb35bb991', description='null'}-172.25.164.83:3717 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Updating cluster description to {type=REPLICA_SET, servers=[{address=172.25.164.82:3717, type=REPLICA_SET_PRIMARY, roundTripTime=2.0 ms, state=CONNECTED}, {address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, roundTripTime=2.5 ms, state=CONNECTED}] 2018-11-14 23:59:55,510 DEBUG jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.connection Closing connection connectionId{localValue:26} 2018-11-14 23:59:55,511 INFO jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.info(SLF4JLogger.java:71) ~ SLF4JLogger.java org.mongodb.driver.connection Closed connection [connectionId{localValue:26}] to 172.25.164.82:3717 because there was a socket exception raised by this connection. 2018-11-14 23:59:55,511 DEBUG jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.cluster Updating cluster description to {type=REPLICA_SET, servers=[{address=172.25.164.82:3717, type=UNKNOWN, state=CONNECTING}, {address=172.25.164.83:3717, type=REPLICA_SET_SECONDARY, roundTripTime=2.5 ms, state=CONNECTED}] 2018-11-14 23:59:55,511 DEBUG jobs-thread-3 com.mongodb.diagnostics.logging.SLF4JLogger.debug(SLF4JLogger.java:56) ~ SLF4JLogger.java org.mongodb.driver.connection Invalidating the connection pool 2018-11-14 23:59:55,511 ERROR jobs-thread-3 task.base.JobExecuteTemplate.doJobInner(JobExecuteTemplate.java:58) ~ JobExecuteTemplate.java TASK >>>>>>>>>>> Job [HandleExpressStateRecordJob] enconter unknow exception