Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-69905

Improve ServiceExecutorSynchronous worker threads

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Service Arch
    • Service Arch 2022-12-26, Service Arch 2022-10-17, Service Arch 2022-11-14, Service Arch 2022-11-28, Service Arch 2022-12-12, Service Arch 2023-01-09

      ServiceExecutorSynchronous creates worker threads when a client calls schedule on it to initiate a task chain. If the task calls schedule while it's executing, that task is enqueued to the worker thread. The worker stops when the queue is empty.

      Issues with this:

      • Uses pthread instead of stdx::thread, so we have two kinds of threads in the system. This is done to control the stack size, but we should be okay to just use the default stack size on 64-bit systems. The stack size is tweaked as a canary for finding deep stack usage in tests, but I think this benefit is very small and we could measure high water mark in other ways.
      • Worker threads are detached, which makes synchronization and thread_local management very difficult. It also means they can run during process exit and can cause interference between unit test scenarios. Detached threads should be avoided if possible. We're already trying to synchronize on the end of these threads' callable function in order to update stats and join the executor, so detaching is kind of pointless.
      • (sub-task SERVER-70145) The biggest issue is that this executor makes a fresh thread for each toplevel task. The contract of this scheduler is that a toplevel task gets a dedicated thread, but it is not promising a FRESH brand new thread, only a dedicated one. So when the toplevel task completes, the worker thread should be reusable. But we don't reuse it. We let it exit and just make new threads for each connection. This seems extremely wasteful and we should just pool these workers and hand out leases to them. Essentially this is not so different from an ordinary ThreadPool except that the threads are different kinds of objects from the stdx::thread objects managed by ThreadPool.
      • We could be using boost::thread for the stack size control instead of raw pthreads.

            Assignee:
            backlog-server-servicearch [DO NOT USE] Backlog - Service Architecture
            Reporter:
            billy.donahue@mongodb.com Billy Donahue
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: