Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-57

Can't save MongoRDD as tempTable

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 0.3
    • Affects Version/s: None
    • Component/s: None
    • None

      Sorry - me again. Things going very well - I just pulled data from an aggregate pipeline into a MongoRDD and I can see the results with .toDF.show() and looks as expected. Also I can printSchema and it looks correct.But when I do this
      aggBillingStatusDf.registerTempTable("temp_billingupdate") it looks successful but when I look for the table like this sqlContext.sql("select * from temp_aggregatedBillingStats").show() it's nowhere to be found. If I showTables , sqlContext.sql("show tables").show(), I still don't see it.

      Being clever, and because I really need the data I tried to take it there like this:
      aggBillingStatusDf.write.mode(SaveMode.Overwrite).saveAsTable("localBillingTest")

      and got this error
      java.lang.RuntimeException: Tables created with SQLContext must be TEMPORARY. Use a HiveContext instead.

      but of course my sqlContext IS a HiveContext:
      scala> sqlContext
      res44: org.apache.spark.sql.SQLContext = org.apache.spark.sql.hive.HiveContext@64ef0650

      Any ideas?

            Assignee:
            Unassigned Unassigned
            Reporter:
            rmwilliams Richard Williams
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: