Uploaded image for project: 'MongoDB Shell'
  1. MongoDB Shell
  2. MONGOSH-1029

Cannot use timestamp from oplog for filtering

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Major - P3 Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: CRUD
    • Environment:
      OS:
      node.js / npm versions:
      Additional info:
    • Not Needed

      Problem Statement/Rationale

      Sorry if I choose the wrong project!

      I have a stream of oplog messages produced by MongoDB Kafka connector. I want to tell the connector to start producing messages from a given point in time. I can do it using pipelines [documented here|https://docs.mongodb.com/kafka-connector/current/kafka-source/.]

      I'm trying to prepare a query that will use the `clusterTime` for this operation but with no luck. The clusterTime is a BSON Timestamp. I'm using MongoDB v4.0.9.

      Steps to Reproduce

      db.gustotest.insertMany([{ 
          "operationType" : "insert", 
          "clusterTime" : {
              "$timestamp" : {
                  "t" : 1634824102.0, 
                  "i" : 1.0
              }
          }, 
          "fullDocument" : {
              "_id" : {
                  "$oid" : "61716fa62b7ffb4a2e01a235"
              }, 
              "location" : "Location01", 
              "organizationID" : "123", 
              "created" : {
                  "$date" : 1634824102357.0
              }
          }, 
          "ns" : {
              "db" : "warehouse", 
              "coll" : "gustotest"
          }, 
          "documentKey" : {
              "_id" : {
                  "$oid" : "61716fa62b7ffb4a2e01a235"
              }
          }
      }
      ])
      

      when we have at least one element, we can give a try and filter by the clusterTime. I've tried many approaches like

       db.gustotest.aggregate( [
         { $addFields:
            {
              convertedDate: { $toDate: {$dateToString:{date:"$clusterTime"}} },
             }
          }
      ] )
      

      or

      db.gustotest.aggregate( [
         { $match:
            {
              clusterTime: { $gt: Timestamp(0, 0) },
             }
          }
      ] ) 

      Is there any other way I can achive this without using $getField, $function that are avaiable in newest versions?

      Expected Results

      Return some results

      Actual Results

      No results or an error.

            Assignee:
            Unassigned Unassigned
            Reporter:
            bartlomiej.klimczak88@gmail.com Bartłomiej Klimczak
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: