contrib/benchmarks/__init__.py
author Augie Fackler <augie@google.com>
Fri, 06 Mar 2020 13:27:41 -0500
changeset 44452 9d2b2df2c2ba
parent 43076 2372284d9457
child 47437 7a430116f639
permissions -rw-r--r--
cleanup: run pyupgrade on our source tree to clean up varying things Built with: hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/**' | xargs pyupgrade --keep-percent-format --keep-extraneous-parens and then blackened. pyupgrade comes from https://github.com/asottile/pyupgrade with a patch to let me preserve extraneous parens (which we use for marking strings that shouldn't be translated), and lets us clean up a bunch of idioms that have cruftily accumulated over the years. # skip-blame no-op automated code cleanups Differential Revision: https://phab.mercurial-scm.org/D8255
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     1
# __init__.py - asv benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     2
#
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     3
# Copyright 2016 Logilab SA <contact@logilab.fr>
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     4
#
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     7
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
     8
# "historical portability" policy of contrib/benchmarks:
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
     9
#
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    10
# We have to make this code work correctly with current mercurial stable branch
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    11
# and if possible with reasonable cost with early Mercurial versions.
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    12
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    13
'''ASV (https://asv.readthedocs.io) benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    14
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    15
Benchmark are parameterized against reference repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    16
directory pointed by the REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    17
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    18
Invocation example:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    19
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    20
    $ export REPOS_DIR=~/hgperf/repos
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    21
    # run suite on given revision
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    22
    $ asv --config contrib/asv.conf.json run REV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    23
    # run suite on new changesets found in stable and default branch
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    24
    $ asv --config contrib/asv.conf.json run NEW
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    25
    # display a comparative result table of benchmark results between two given
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    26
    # revisions
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    27
    $ asv --config contrib/asv.conf.json compare REV1 REV2
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    28
    # compute regression detection and generate ASV static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    29
    $ asv --config contrib/asv.conf.json publish
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    30
    # serve the static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    31
    $ asv --config contrib/asv.conf.json preview
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    32
'''
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    33
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    34
from __future__ import absolute_import
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    35
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    36
import functools
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    37
import os
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    38
import re
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    39
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    40
from mercurial import (
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    41
    extensions,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    42
    hg,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    43
    ui as uimod,
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    44
    util,
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    45
)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    46
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    47
basedir = os.path.abspath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    48
    os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    49
)
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    50
reposdir = os.environ['REPOS_DIR']
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    51
reposnames = [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    52
    name
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    53
    for name in os.listdir(reposdir)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    54
    if os.path.isdir(os.path.join(reposdir, name, ".hg"))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    55
]
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    56
if not reposnames:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    57
    raise ValueError("No repositories found in $REPO_DIR")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    58
outputre = re.compile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    59
    (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    60
        r'! wall (\d+.\d+) comb \d+.\d+ user \d+.\d+ sys '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    61
        r'\d+.\d+ \(best of \d+\)'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    62
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    63
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    64
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    65
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    66
def runperfcommand(reponame, command, *args, **kwargs):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    67
    os.environ["HGRCPATH"] = os.environ.get("ASVHGRCPATH", "")
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    68
    # for "historical portability"
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    69
    # ui.load() has been available since d83ca85
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    70
    if util.safehasattr(uimod.ui, "load"):
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    71
        ui = uimod.ui.load()
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    72
    else:
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    73
        ui = uimod.ui()
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    74
    repo = hg.repository(ui, os.path.join(reposdir, reponame))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    75
    perfext = extensions.load(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    76
        ui, 'perfext', os.path.join(basedir, 'contrib', 'perf.py')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    77
    )
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    78
    cmd = getattr(perfext, command)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    79
    ui.pushbuffer()
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    80
    cmd(ui, repo, *args, **kwargs)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    81
    output = ui.popbuffer()
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    82
    match = outputre.search(output)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    83
    if not match:
44452
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    84
        raise ValueError("Invalid output {}".format(output))
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    85
    return float(match.group(1))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    86
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
    87
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    88
def perfbench(repos=reposnames, name=None, params=None):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    89
    """decorator to declare ASV benchmark based on contrib/perf.py extension
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    90
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    91
    An ASV benchmark is a python function with the given attributes:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    92
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    93
    __name__: should start with track_, time_ or mem_ to be collected by ASV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    94
    params and param_name: parameter matrix to display multiple graphs on the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    95
    same page.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    96
    pretty_name: If defined it's displayed in web-ui instead of __name__
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    97
    (useful for revsets)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    98
    the module name is prepended to the benchmark name and displayed as
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    99
    "category" in webui.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   100
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   101
    Benchmarks are automatically parameterized with repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   102
    REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   103
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   104
    `params` is the param matrix in the form of a list of tuple
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   105
    (param_name, [value0, value1])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   106
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   107
    For example [(x, [a, b]), (y, [c, d])] declare benchmarks for
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   108
    (a, c), (a, d), (b, c) and (b, d).
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   109
    """
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   110
    params = list(params or [])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   111
    params.insert(0, ("repo", repos))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   112
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   113
    def decorator(func):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   114
        @functools.wraps(func)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   115
        def wrapped(repo, *args):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   116
            def perf(command, *a, **kw):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   117
                return runperfcommand(repo, command, *a, **kw)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
   118
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   119
            return func(perf, *args)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   120
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   121
        wrapped.params = [p[1] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   122
        wrapped.param_names = [p[0] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   123
        wrapped.pretty_name = name
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   124
        return wrapped
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 30588
diff changeset
   125
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   126
    return decorator