contrib/benchmarks/__init__.py
author Arseniy Alekseyev <aalekseyev@janestreet.com>
Tue, 09 Apr 2024 11:12:24 +0100
changeset 51602 68929cf3c0c6
parent 50928 d718eddf01d9
permissions -rw-r--r--
match: avoid rust fast path if the matcher was tampered with Otherwise the fast path does not respect the modifications made by the extension (concretely largefiles, but other extensions can start using that too)
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     1
# __init__.py - asv benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     2
#
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     3
# Copyright 2016 Logilab SA <contact@logilab.fr>
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     4
#
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     7
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
     8
# "historical portability" policy of contrib/benchmarks:
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
     9
#
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    10
# We have to make this code work correctly with current mercurial stable branch
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    11
# and if possible with reasonable cost with early Mercurial versions.
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    12
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    13
'''ASV (https://asv.readthedocs.io) benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    14
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    15
Benchmark are parameterized against reference repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    16
directory pointed by the REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    17
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    18
Invocation example:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    19
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    20
    $ export REPOS_DIR=~/hgperf/repos
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    21
    # run suite on given revision
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    22
    $ asv --config contrib/asv.conf.json run REV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    23
    # run suite on new changesets found in stable and default branch
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    24
    $ asv --config contrib/asv.conf.json run NEW
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    25
    # display a comparative result table of benchmark results between two given
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    26
    # revisions
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    27
    $ asv --config contrib/asv.conf.json compare REV1 REV2
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    28
    # compute regression detection and generate ASV static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    29
    $ asv --config contrib/asv.conf.json publish
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    30
    # serve the static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    31
    $ asv --config contrib/asv.conf.json preview
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    32
'''
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    33
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    34
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    35
import functools
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    36
import os
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    37
import re
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    38
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    39
from mercurial import (
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    40
    extensions,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    41
    hg,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    42
    ui as uimod,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    43
)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    44
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    45
basedir = os.path.abspath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    46
    os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    47
)
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    48
reposdir = os.environ['REPOS_DIR']
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    49
reposnames = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    50
    name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    51
    for name in os.listdir(reposdir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    52
    if os.path.isdir(os.path.join(reposdir, name, ".hg"))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    53
]
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    54
if not reposnames:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    55
    raise ValueError("No repositories found in $REPO_DIR")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    56
outputre = re.compile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    57
    (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    58
        r'! wall (\d+.\d+) comb \d+.\d+ user \d+.\d+ sys '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    59
        r'\d+.\d+ \(best of \d+\)'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    60
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    61
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    62
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    63
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    64
def runperfcommand(reponame, command, *args, **kwargs):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    65
    os.environ["HGRCPATH"] = os.environ.get("ASVHGRCPATH", "")
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    66
    # for "historical portability"
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    67
    # ui.load() has been available since d83ca85
50928
d718eddf01d9 safehasattr: drop usage in favor of hasattr
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
    68
    if hasattr(uimod.ui, "load"):
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    69
        ui = uimod.ui.load()
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    70
    else:
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    71
        ui = uimod.ui()
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    72
    repo = hg.repository(ui, os.path.join(reposdir, reponame))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    73
    perfext = extensions.load(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    74
        ui, 'perfext', os.path.join(basedir, 'contrib', 'perf.py')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    75
    )
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    76
    cmd = getattr(perfext, command)
47458
50cd14dbd3b3 benchmarks: restore `output` variable lost in D10884
Martin von Zweigbergk <martinvonz@google.com>
parents: 47437
diff changeset
    77
    ui.pushbuffer()
50cd14dbd3b3 benchmarks: restore `output` variable lost in D10884
Martin von Zweigbergk <martinvonz@google.com>
parents: 47437
diff changeset
    78
    cmd(ui, repo, *args, **kwargs)
50cd14dbd3b3 benchmarks: restore `output` variable lost in D10884
Martin von Zweigbergk <martinvonz@google.com>
parents: 47437
diff changeset
    79
    output = ui.popbuffer()
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    80
    match = outputre.search(output)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    81
    if not match:
44452
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    82
        raise ValueError("Invalid output {}".format(output))
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    83
    return float(match.group(1))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    84
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    85
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    86
def perfbench(repos=reposnames, name=None, params=None):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    87
    """decorator to declare ASV benchmark based on contrib/perf.py extension
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    88
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    89
    An ASV benchmark is a python function with the given attributes:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    90
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    91
    __name__: should start with track_, time_ or mem_ to be collected by ASV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    92
    params and param_name: parameter matrix to display multiple graphs on the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    93
    same page.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    94
    pretty_name: If defined it's displayed in web-ui instead of __name__
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    95
    (useful for revsets)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    96
    the module name is prepended to the benchmark name and displayed as
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    97
    "category" in webui.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    98
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    99
    Benchmarks are automatically parameterized with repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   100
    REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   101
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   102
    `params` is the param matrix in the form of a list of tuple
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   103
    (param_name, [value0, value1])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   104
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   105
    For example [(x, [a, b]), (y, [c, d])] declare benchmarks for
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   106
    (a, c), (a, d), (b, c) and (b, d).
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   107
    """
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   108
    params = list(params or [])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   109
    params.insert(0, ("repo", repos))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   110
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   111
    def decorator(func):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   112
        @functools.wraps(func)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   113
        def wrapped(repo, *args):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   114
            def perf(command, *a, **kw):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   115
                return runperfcommand(repo, command, *a, **kw)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
   116
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   117
            return func(perf, *args)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   118
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   119
        wrapped.params = [p[1] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   120
        wrapped.param_names = [p[0] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   121
        wrapped.pretty_name = name
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   122
        return wrapped
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
   123
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   124
    return decorator