contrib/perf-utils/perf-revlog-write-plot.py
author Manuel Jacob <me@manueljacob.de>
Mon, 11 Jul 2022 01:51:20 +0200
branchstable
changeset 49378 094a5fa3cf52
parent 48875 6000f5b25c9b
permissions -rwxr-xr-x
procutil: make stream detection in make_line_buffered more correct and strict In make_line_buffered(), we don’t want to wrap the stream if we know that lines get flushed to the underlying raw stream already. Previously, the heuristic was too optimistic. It assumed that any stream which is not an instance of io.BufferedIOBase doesn’t need wrapping. However, there are buffered streams that aren’t instances of io.BufferedIOBase, like Mercurial’s own winstdout. The new logic is different in two ways: First, only for the check, if unwraps any combination of WriteAllWrapper and winstdout. Second, it skips wrapping the stream only if it is an instance of io.RawIOBase (or already wrapped). If it is an instance of io.BufferedIOBase, it gets wrapped. In any other case, the function raises an exception. This ensures that, if an unknown stream is passed or we add another wrapper in the future, we don’t wrap the stream if it’s already line buffered or not wrap the stream if it’s not line buffered. In fact, this was already helpful during development of this change. Without it, I possibly would have forgot that WriteAllWrapper needs to be ignored for the check, leading to unnecessary wrapping if stdout is unbuffered. The alternative would have been to always wrap unknown streams. However, I don’t think that anyone would benefit from being less strict. We can expect streams from the standard library to be subclassing either io.RawIOBase or io.BufferedIOBase, so running Mercurial in the standard way should not regress by this change. Py2exe might replace sys.stdout and sys.stderr, but that currently breaks Mercurial anyway and also these streams don’t claim to be interactive, so this function is not called for them.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
45830
c102b704edb5 global: use python3 in shebangs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
     1
#!/usr/bin/env python3
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     2
#
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     3
#  Copyright 2018 Paul Morelle <Paul.Morelle@octobus.net>
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     4
#
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     7
#
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     8
# This script use the output of `hg perfrevlogwrite -T json --details` to draw
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
     9
# various plot related to write performance in a revlog
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    10
#
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    11
# usage: perf-revlog-write-plot.py details.json
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    12
import json
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    13
import re
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    14
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    15
import numpy as np
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    16
import scipy.signal
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    17
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    18
from matplotlib import (
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    19
    pyplot as plt,
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    20
    ticker as mticker,
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    21
)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    22
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    23
41190
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
    24
def plot(data, title=None):
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    25
    items = {}
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    26
    re_title = re.compile(r'^revisions #\d+ of \d+, rev (\d+)$')
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    27
    for item in data:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    28
        m = re_title.match(item['title'])
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    29
        if m is None:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    30
            continue
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    31
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    32
        rev = int(m.group(1))
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    33
        items[rev] = item
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    34
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    35
    min_rev = min(items.keys())
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    36
    max_rev = max(items.keys())
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    37
    ary = np.empty((2, max_rev - min_rev + 1))
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    38
    for rev, item in items.items():
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    39
        ary[0][rev - min_rev] = rev
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    40
        ary[1][rev - min_rev] = item['wall']
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    41
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    42
    fig = plt.figure()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    43
    comb_plt = fig.add_subplot(211)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    44
    other_plt = fig.add_subplot(212)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    45
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    46
    comb_plt.plot(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    47
        ary[0], np.cumsum(ary[1]), color='red', linewidth=1, label='comb'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    48
    )
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    49
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    50
    plots = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    51
    p = other_plt.plot(ary[0], ary[1], color='red', linewidth=1, label='wall')
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    52
    plots.append(p)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    53
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    54
    colors = {
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    55
        10: ('green', 'xkcd:grass green'),
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    56
        100: ('blue', 'xkcd:bright blue'),
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    57
        1000: ('purple', 'xkcd:dark pink'),
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    58
    }
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    59
    for n, color in colors.items():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    60
        avg_n = np.convolve(ary[1], np.full(n, 1.0 / n), 'valid')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    61
        p = other_plt.plot(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    62
            ary[0][n - 1 :],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    63
            avg_n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    64
            color=color[0],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    65
            linewidth=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    66
            label='avg time last %d' % n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    67
        )
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    68
        plots.append(p)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    69
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    70
        med_n = scipy.signal.medfilt(ary[1], n + 1)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    71
        p = other_plt.plot(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    72
            ary[0],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    73
            med_n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    74
            color=color[1],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    75
            linewidth=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    76
            label='median time last %d' % n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
    77
        )
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    78
        plots.append(p)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    79
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    80
    formatter = mticker.ScalarFormatter()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    81
    formatter.set_scientific(False)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    82
    formatter.set_useOffset(False)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    83
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    84
    comb_plt.grid()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    85
    comb_plt.xaxis.set_major_formatter(formatter)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    86
    comb_plt.legend()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    87
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    88
    other_plt.grid()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    89
    other_plt.xaxis.set_major_formatter(formatter)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    90
    leg = other_plt.legend()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    91
    leg2plot = {}
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    92
    for legline, plot in zip(leg.get_lines(), plots):
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    93
        legline.set_picker(5)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    94
        leg2plot[legline] = plot
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    95
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    96
    def onpick(event):
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    97
        legline = event.artist
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    98
        plot = leg2plot[legline]
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
    99
        visible = not plot[0].get_visible()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   100
        for l in plot:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   101
            l.set_visible(visible)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   102
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   103
        if visible:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   104
            legline.set_alpha(1.0)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   105
        else:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   106
            legline.set_alpha(0.2)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   107
        fig.canvas.draw()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
   108
41190
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
   109
    if title is not None:
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
   110
        fig.canvas.set_window_title(title)
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   111
    fig.canvas.mpl_connect('pick_event', onpick)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   112
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   113
    plt.show()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   114
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   115
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   116
if __name__ == '__main__':
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   117
    import sys
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   118
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   119
    if len(sys.argv) > 1:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   120
        print('reading from %r' % sys.argv[1])
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   121
        with open(sys.argv[1], 'r') as fp:
41190
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
   122
            plot(json.load(fp), title=sys.argv[1])
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   123
    else:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   124
        print('reading from stdin')
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
   125
        plot(json.load(sys.stdin))