Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30111

Mapreduce uses global lock when outputting to empty collection

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: 3.2.15, 3.4.6, 3.5.9
    • Component/s: MapReduce
    • None
    • ALL
    • Hide

      Create a mapreduce and output to an empty collection, and it will get the global lock.

      Show
      Create a mapreduce and output to an empty collection, and it will get the global lock.

      I want to efficiently mapreduce some data and output to an empty collection without touching the global lock.

      There is an optimization in mr.cpp in postProcessCollectionNonAtomic which forces "replace" mode on outputting to empty collections.
      Unfortunately, "replace" mode gets a global lock, even when writing to the same database.

      Both of these are unexpected.

      The workaround is to insert a dummy document into the empty collection, and output using "merge" mode. This is unintuitive and requires digging into the mongo core code.

      There is a blog post describing the issue and workaround here:
      http://venublog.com/2014/04/05/mongodb-map-reduce-how-to-avoid-global-locks

      The blog post is a few years old but this is still an issue in the master branch:
      https://github.com/mongodb/mongo/blob/master/src/mongo/db/commands/mr.cpp#L643

            Assignee:
            Unassigned Unassigned
            Reporter:
            james@rabb.it James Mitchell
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: