Uploaded image for project: 'Go Driver'
  1. Go Driver
  2. GODRIVER-3255

Heartbeat flood if K_SERVICE or similar env is set

    • Go Drivers
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      Detailed steps to reproduce the problem?

      package main
      
      import (
      	"context"
      	"fmt"
      	"sync/atomic"
      	"time"
      
      	"go.mongodb.org/mongo-driver/event"
      	"go.mongodb.org/mongo-driver/mongo"
      	"go.mongodb.org/mongo-driver/mongo/options"
      )
      
      func main() {
      	var heartbeatsStartedCounter atomic.Int64
      
      	now := time.Now()
      
      	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
      	defer cancel()
      	client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://localhost:27017").SetServerMonitor(&event.ServerMonitor{
      		ServerHeartbeatStarted: func(_ *event.ServerHeartbeatStartedEvent) {
      			heartbeatsStartedCounter.Add(1)
      		},
      	}))
      
      	if err != nil {
      		panic(err)
      	}
      
      	defer client.Disconnect(ctx)
      
      	ticker := time.Tick(1 * time.Second)
      	for {
      		fmt.Printf("[%05d] total heartbeats until now: %d\n", int(time.Since(now).Seconds()), heartbeatsStartedCounter.Load())
      		<-ticker
      	}
      }
      

      Normal Execution

      $ go run .
      [00000] total heartbeats until now: 0
      [00001] total heartbeats until now: 2
      [00002] total heartbeats until now: 2
      [00003] total heartbeats until now: 2
      [00004] total heartbeats until now: 2
      [00005] total heartbeats until now: 2
      [00006] total heartbeats until now: 2
      [00007] total heartbeats until now: 2
      [00008] total heartbeats until now: 2
      [00009] total heartbeats until now: 2
      [00010] total heartbeats until now: 2
      [00011] total heartbeats until now: 3
      [00012] total heartbeats until now: 3
      [00013] total heartbeats until now: 3
      [00014] total heartbeats until now: 3
      [00015] total heartbeats until now: 3
      [00016] total heartbeats until now: 3
      [00017] total heartbeats until now: 3
      [00018] total heartbeats until now: 3
      [00019] total heartbeats until now: 3
      [00020] total heartbeats until now: 3
      [00021] total heartbeats until now: 4
      [00022] total heartbeats until now: 4
      [00023] total heartbeats until now: 4
      [00024] total heartbeats until now: 4
      

      Execution if K_SERVICE is set

      $ K_SERVICE=test go run .
      [00000] total heartbeats until now: 0
      [00001] total heartbeats until now: 20
      [00002] total heartbeats until now: 49
      [00003] total heartbeats until now: 63
      [00004] total heartbeats until now: 81
      [00005] total heartbeats until now: 94
      [00006] total heartbeats until now: 107
      [00007] total heartbeats until now: 117
      [00008] total heartbeats until now: 128
      [00009] total heartbeats until now: 138
      [00010] total heartbeats until now: 148
      [00011] total heartbeats until now: 157
      [00012] total heartbeats until now: 173
      [00013] total heartbeats until now: 190
      [00014] total heartbeats until now: 210
      [00015] total heartbeats until now: 230
      [00016] total heartbeats until now: 253
      [00017] total heartbeats until now: 274
      [00018] total heartbeats until now: 298
      [00019] total heartbeats until now: 319
      [00020] total heartbeats until now: 343
      [00021] total heartbeats until now: 365
      [00022] total heartbeats until now: 387
      [00023] total heartbeats until now: 400
      [00024] total heartbeats until now: 416
      

      As you can server, there are way to many heartbeats triggered.
      This should be the same for any env var listed here: https://github.com/mongodb/mongo-go-driver/blob/v1.15.1/x/mongo/driver/operation/hello.go#L145-L152

      Root Cause:

      The driver decides to skip waitUntilNextCheck based on isStreamable(s): https://github.com/mongodb/mongo-go-driver/blob/v1.15.1/x/mongo/driver/topology/server.go#L669-L671

      But in the check code it will only use streaming if isStreamingEnabled(s) && isStreamable(s) https://github.com/mongodb/mongo-go-driver/blob/v1.15.1/x/mongo/driver/topology/server.go#L855

      isStreamingEnabled will return false if K_SERVICE is set: https://github.com/mongodb/mongo-go-driver/blob/v1.15.1/x/mongo/driver/topology/server.go#L809 (if serverMonitoringMode is not set)

      The same bug also occurs if serverMonitoringMode is set to poll.

      I guess this can be best solved by creating a useStreamingForHeartbeat function, which can be used in both pleaces and prevent similar bugs in the future.

      Workaround until this is fixed: SetServerMonitoringMode(connstring.ServerMonitoringModeStream)

      Definition of done: what must be done to consider the task complete?

      The exact Go version used, with patch level:

      $ go version
      go version go1.22.2 linux/amd64

      The exact version of the Go driver used:

      $ go list -m go.mongodb.org/mongo-driver
      go.mongodb.org/mongo-driver v1.15.1

      Describe how MongoDB is set up. Local vs Hosted, version, topology, load balanced, etc.

      The operating system and version (e.g. Windows 7, OSX 10.8, ...)

      Ubuntu 24.04 (Laptop used for reproduction) and Ubuntu 22.04 (Kubernetes Node OS)

      Security Vulnerabilities

      If you’ve identified a security vulnerability in a driver or any other MongoDB project, please report it according to the instructions here
      No

            Assignee:
            preston.vasquez@mongodb.com Preston Vasquez
            Reporter:
            philipp.stehle@kbst-gmbh.de Philipp Stehle
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: