Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24019

Eviction failure because of hazard pointer and query throughput regression (not DML)

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.5
    • Component/s: WiredTiger
    • None
    • Fully Compatible
    • ALL

      Query throughput is decreased temporarily.
      And during this issue, eviction failure and hazard pointer eviction blocked counter is excessively increased.

      Only query throughput is decreased, not Update and Delete Insert.

      insert query update delete getmore command % dirty % used flushes vsize   res  qr|qw   ar|aw netIn netOut conn         set repl                      time
    *0  2115   1337     20     287   778|0     0.3   80.4       0 43.2G 35.5G    3|0    11|2    2m    34m  750 testreplset  PRI 2016-05-02T21:26:28+09:00
    *0  2149   1386     16     290   794|0     0.4   80.9       0 43.2G 35.5G    0|1     0|0    2m    35m  750 testreplset  PRI 2016-05-02T21:26:29+09:00
    *0  1958   1167     13     275   751|0     0.3   80.3       0 43.2G 35.5G    0|0     0|0    1m    31m  750 testreplset  PRI 2016-05-02T21:26:30+09:00
    *0  1931   1253     19     259   548|0     0.5   80.8       0 43.2G 35.5G    0|0     4|1    1m    31m  750 testreplset  PRI 2016-05-02T21:26:31+09:00
    *0  2156   1329     20     279   791|0     0.4   80.8       0 43.2G 35.5G    0|0     0|0    2m    35m  750 testreplset  PRI 2016-05-02T21:26:32+09:00
    *0  2070   1188     13     297   806|0     0.3   80.2       0 43.2G 35.5G    0|0    14|7    1m    33m  750 testreplset  PRI 2016-05-02T21:26:33+09:00
    *0  2095   1299     20     297   761|0     0.5   80.8       1 43.2G 35.5G    0|1    92|2    1m    32m  750 testreplset  PRI 2016-05-02T21:26:34+09:00
    *0  2030   1305     31     304   838|0     0.2   80.7       0 43.2G 35.5G    0|0     1|1    2m    34m  750 testreplset  PRI 2016-05-02T21:26:35+09:00
    *0  2047   1136      9     276   803|0     0.1   80.4       0 43.2G 35.5G    0|0     1|0    1m    33m  750 testreplset  PRI 2016-05-02T21:26:36+09:00
    *0  1477   1310     22     112   331|0     0.1   80.5       0 43.2G 35.5G   72|0  128|35    1m    21m  750 testreplset  PRI 2016-05-02T21:26:37+09:00
    *0  1090   1365     13      31   132|0     0.1   80.1       0 43.2G 35.5G  105|0  128|52  893k    17m  751 testreplset  PRI 2016-05-02T21:26:38+09:00
    *0   969   1205     14      22    92|0     0.1   80.2       0 43.2G 35.5G  106|0  128|44  806k    16m  753 testreplset  PRI 2016-05-02T21:26:39+09:00
    *0   981   1249     15      21    95|0     0.1   80.2       0 43.2G 35.5G  112|0  128|48  814k    16m  755 testreplset  PRI 2016-05-02T21:26:40+09:00
    *0   883   1370     23      22    95|0     0.2   80.4       0 43.2G 35.5G  118|1  128|72  819k    15m  756 testreplset  PRI 2016-05-02T21:26:41+09:00
    *0   780   1123     15      18    75|0     0.1   80.0       0 43.2G 35.5G  117|0  128|44  732k    13m  756 testreplset  PRI 2016-05-02T21:26:42+09:00
    *0   992   1288     21      19    83|0     0.1   80.1       0 43.2G 35.5G  118|0  128|46  833k    16m  758 testreplset  PRI 2016-05-02T21:26:43+09:00
    *0   810   1287     11      21    85|0     0.2   80.4       0 43.2G 35.5G  111|1  128|50  779k    13m  758 testreplset  PRI 2016-05-02T21:26:44+09:00
    *0   885   1177      9      21    93|0     0.2   80.2       0 43.2G 35.5G  116|2  128|31  780k    15m  758 testreplset  PRI 2016-05-02T21:26:45+09:00
    *0   825   1239     20      21    94|0     0.2   80.3       0 43.2G 35.5G  122|0  127|47  770k    14m  759 testreplset  PRI 2016-05-02T21:26:46+09:00
    *0   653   1259     18      19    88|0     0.2   80.2       0 43.2G 35.5G  104|1  128|35  741k    11m  759 testreplset  PRI 2016-05-02T21:26:47+09:00

      I am not sure this is related with SERVER-23778.
      And I have experienced this issue from patched (https://github.com/mongodb/mongo/pull/1078) mongodb.

        1. sharding_status.txt
          28 kB
        2. pages_eviceted_by.png
          pages_eviceted_by.png
          121 kB
        3. mongostat.txt
          154 kB
        4. mongostat.png
          mongostat.png
          242 kB
        5. forcible_eviction.png
          forcible_eviction.png
          91 kB
        6. eviction_failed.png
          eviction_failed.png
          95 kB
        7. eviction_blocked.png
          eviction_blocked.png
          92 kB
        8. diagnostic_data.tar.gz
          25.64 MB
        9. block_reads_writes.png
          block_reads_writes.png
          120 kB

            Assignee:
            michael.cahill@mongodb.com Michael Cahill (Inactive)
            Reporter:
            sunguck.lee@gmail.com 아나 하리
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: