Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-121

Why my quite simple scala test won't stop or print result

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.0.0
    • Component/s: API
    • None
    • Environment:
      mongodb: 3.x
      spark: 2.0.1
      scala: 2.11.8 

      class MongoSpec extends FunSpec{

  val mongoUri = "mongodb://192.168.0.240/LocalStorage.ShardFile"
  val input = "spark.mongodb.input.uri"
  val spark = SparkSession.builder().master("local").appName("mongo")
    .config(input, mongoUri)
    .getOrCreate()

  describe("read first"){
    it("should return 1 line"){
      val rdd = MongoSpark.load(spark)
      println("first line: " + rdd.first)
    }
  }
}

      the prints like the followings(a lot of Checking status and update cluster):

      [17-05-08 17:54:35.070] [DEBUG]{org.mongodb.driver.protocol.command ScalaTest-run-running-MongoSpec} Sending command {buildInfo : BsonInt32{value=1}} to database LocalStorage on connection [connectionId{localValue:2, serverValue:2542}] to server 192.168.0.240:27017
[17-05-08 17:54:35.072] [DEBUG]{org.mongodb.driver.protocol.command ScalaTest-run-running-MongoSpec} Command execution completed
[17-05-08 17:54:35.076] [DEBUG]{org.mongodb.driver.protocol.command ScalaTest-run-running-MongoSpec} Sending command {collStats : BsonString{value='ShardFile'}} to database LocalStorage on connection [connectionId{localValue:2, serverValue:2542}] to server 192.168.0.240:27017
[17-05-08 17:54:35.081] [DEBUG]{org.mongodb.driver.protocol.command ScalaTest-run-running-MongoSpec} Command execution completed
[17-05-08 17:54:35.086] [DEBUG]{org.mongodb.driver.protocol.command ScalaTest-run-running-MongoSpec} Sending command {aggregate : BsonString{value='ShardFile'}} to database LocalStorage on connection [connectionId{localValue:2, serverValue:2542}] to server 192.168.0.240:27017
[17-05-08 17:54:40.349] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Checking status of 192.168.0.240:27017
[17-05-08 17:54:40.350] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Updating cluster description to  {type=STANDALONE, servers=[{address=192.168.0.240:27017, type=STANDALONE, roundTripTime=1.7 ms, state=CONNECTED}]
[17-05-08 17:54:50.351] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Checking status of 192.168.0.240:27017
[17-05-08 17:54:50.352] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Updating cluster description to  {type=STANDALONE, servers=[{address=192.168.0.240:27017, type=STANDALONE, roundTripTime=1.5 ms, state=CONNECTED}]
[17-05-08 17:55:00.353] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Checking status of 192.168.0.240:27017
[17-05-08 17:55:00.354] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Updating cluster description to  {type=STANDALONE, servers=[{address=192.168.0.240:27017, type=STANDALONE, roundTripTime=1.4 ms, state=CONNECTED}]
[17-05-08 17:55:10.354] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Checking status of 192.168.0.240:27017
[17-05-08 17:55:10.356] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Updating cluster description to  {type=STANDALONE, servers=[{address=192.168.0.240:27017, type=STANDALONE, roundTripTime=1.4 ms, state=CONNECTED}]
[17-05-08 17:55:20.356] [DEBUG]{org.mongodb.driver.cluster cluster-ClusterId{value='5910402403f4344e80403bd9', description='null'}-192.168.0.240:27017} Checking status of 192.168.0.240:27017

            Assignee:
            Unassigned Unassigned
            Reporter:
            zzzhy zzzhy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: