Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-9079

All memory barriers on ARM should be DMB instead of DSB

    • 5
    • Storage - Ra 2022-04-18, Storage - Ra 2022-05-02
    • v6.0

      Summary
      For the ARM platform switch from using DSB to DMB barriers. DSB is costlier and higher level of synchronization than what WiredTiger requires.

      Motivation
      ARM offers three levels of synchronization barriers, namely:

      1. ISB (Instruction Synchronization Barrier)
      2. DMB (Data Memory Barrier)
      3. DSB (Data Synchronization Barrier)

      They are listed in the order of the cost of using them.

      Here is the relevant WiredTiger code that implements the barriers using the ARM instructions:

      /*
       * dmb are chosen here because they are sufficient to guarantee the ordering described above. We
       * don't want to use dsbs because they provide a much stronger guarantee of completion which isn't
       * required. Additionally, dsbs synchronize other system activities such as tlb and cache
       * maintenance instructions which is not required in this case.
       *
       * A shareability domain of inner-shareable is selected because all the entities participating in
       * the ordering requirements are CPUs and ordering with respect to other devices or memory-types
       * isn't required.
       */
      #define WT_FULL_BARRIER()                         \
          do {                                          \
              __asm__ volatile("dmb ish" ::: "memory"); \
          } while (0)
      #define WT_READ_BARRIER()                           \
          do {                                            \
              __asm__ volatile("dsb ishld" ::: "memory"); \
          } while (0)
      #define WT_WRITE_BARRIER()                          \
          do {                                            \
              __asm__ volatile("dsb ishst" ::: "memory"); \
          } while (0)
      

      As the comment says we do not need the dsb , but the full barrier is dmb while the read/write barriers are dsb.

      The intention of this ticket is to make all the barriers consistent and use the dmb instruction.

      Note: For reference, WT-8959 has more context on the ARM barriers and the atomics.

      • Does this affect any team outside of WT?
        No
      • If the problem does occur, what are the consequences and how severe are they?
        The usage of DSB is likely preventing WiredTiger from getting higher performance. Also, if there are synchronization bugs in the code, dsb is more likely to prevent them from showing up in the testing.
      • Is this issue urgent?
        Yes

      Acceptance Criteria (Definition of Done)
      Switch the read and write barriers to the dmb instruction

      • Testing
      1. WiredTiger stress test on the ARM platforms
      2. WiredTiger perf tests should not show a regression, an improvement is expected
      3. MongoDB patch tests should pass.
      4. Evaluate MongoDB perf tests on ARM. Open question whether after or before merging this change in develop.
      • Documentation update
        None

            Assignee:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Reporter:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: