Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7603

Sort statistics to fix JSON output

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.1, 4.4.8, 5.0.2, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • 5
    • Storage - Ra 2021-07-12
    • v4.4

      I noticed this while trying to get t2 data from test format, a lot of the stats I was interested in were missing.

      It turns out that when we output our statistics JSON we don't sort the sets. So we have duplicate cache, cursor, reconciliation and transaction keys (possibly more) in the JSON. The net effect being that when converted into a proper json by a tool, e.g. t2 the first duplicated key is usually deleted. As such we lose some statistics.

      There's a few possible fixes here but the one I implemented to test is simply sorting the connection statistics (the same will need to be done for the datasource stats):

       diff --git a/dist/stat.py b/dist/stat.py
      index 2e0cb0a86..99624fa91 100644
      --- a/dist/stat.py
      +++ b/dist/stat.py
      @@ -3,13 +3,14 @@ import re, string, sys, textwrap
       from dist import compare_srcfile, format_srcfile
      -
      +from functools import cmp_to_key
       # Read the source files.
       from stat_data import groups, dsrc_stats, connection_stats, conn_dsrc_stats, join_stats, \
      -    session_stats
      +    session_stats, Stat connection_statistics = connection_stats
       connection_statistics.extend(conn_dsrc_stats)
      +connection_statistics = sorted(connection_statistics, key=cmp_to_key(Stat.make_comparator(Stat.__cmp__)))
       dsrc_statistics = dsrc_stats
       dsrc_statistics.extend(conn_dsrc_stats)diff --git a/dist/stat_data.py b/dist/stat_data.py
      index 89a5578b3..2cb718ebb 100644
      --- a/dist/stat_data.py
      +++ b/dist/stat_data.py
      @@ -29,7 +29,17 @@ class Stat:
               self.flags = flags     def __cmp__(self, other):
      -        return cmp(self.desc.lower(), other.desc.lower())
      +        return self.desc.lower() < other.desc.lower()
      +
      +    def make_comparator(less_than):
      +        def compare(x, y):
      +            if less_than(x, y):
      +                return -1
      +            elif less_than(y, x):
      +                return 1
      +            else:
      +                return 0
      +        return compare
      

       

        1. t2 broken.png
          47 kB
          Etienne Petrel
        2. t2 fixed.png
          150 kB
          Etienne Petrel
        3. WiredTigerStat.22.12.broken
          22 kB
          Etienne Petrel
        4. WiredTigerStat.22.13.fixed
          22 kB
          Etienne Petrel

            Assignee:
            etienne.petrel@mongodb.com Etienne Petrel
            Reporter:
            luke.pearson@mongodb.com Luke Pearson
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: